Metraipolis › archive › dc_f58285d29d02

A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methods — arXiv:2609.30397

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.30397
License (as archived)
CC0-1.0
Archived
2026-10-08T01:38:08.939Z
SHA-256 of the archived text
a68d1d97375c65a723ae5d98c3d0d2f9f1d6ef2b0048254a8052292796185dda
Stable URL of this page
https://metraipolis.com/documento/dc_f58285d29d02

How to cite

A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methods — arXiv:2609.30397 — arXiv (en), https://arxiv.org/abs/2609.30397 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f58285d29d02 (Metraipolis archive dc_f58285d29d02, sha256 a68d1d97375c65a723ae5d98c3d0d2f9f1d6ef2b0048254a8052292796185dda)
BibTeX
@misc{dcf58285d29d02,
  title = {A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methods — arXiv:2609.30397},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.30397},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f58285d29d02; sha256 a68d1d97375c65a723ae5d98c3d0d2f9f1d6ef2b0048254a8052292796185dda},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30397. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Evaluating explainable Artificial Intelligence (XAI) methods is a challenging task due to the lack of reliable evaluation procedures and, in particular, the absence of ground truth explanations. In the literature, existing evaluation approaches typically assess explanations by measuring their fidelity with respect to the predictions of a black-box model. However, such evaluation strategies only quantify the degree to which an explanation reproduces the model's output, without ensuring that the explanation correctly reflects the underlying decision process. As a consequence, different explanations may achieve similar fidelity scores while providing inconsistent or misleading interpretations of the model behavior. In this paper, we propose a framework for the evaluation of XAI methods based on synthetic ground truth. The proposed approach relies on controlled interventions to generate synthetic datasets in which the importance of input components can be determined by design. This enables the construction of ground truth explanations that are directly aligned with the behavior of the model under analysis. The framework is instantiated across three data domains, namely binary images, tabular data, and time series, allowing a comprehensive assessment of explanation methods in heterogeneous settings. Experimental results obtained by evaluating nine widely used XAI methods show significant limitations in current techniques and highlight the importance of synthetic, intervention-based benchmarks for a reliable assessment of explanation quality.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.30397 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:12:14.234Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).