Metraipolis › archive › dc_98223ee5171e
EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing — arXiv:2609.17953
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.17953
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T16:16:16.034Z
- SHA-256 of the archived text
7ff0d87cd47071d9f19e99c5eebcc52a5c2f483ad3b89726fc3fcfe7bad92fc7- Stable URL of this page
https://metraipolis.com/documento/dc_98223ee5171e
How to cite
EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing — arXiv:2609.17953 — arXiv (en), https://arxiv.org/abs/2609.17953 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_98223ee5171e (Metraipolis archive dc_98223ee5171e, sha256 7ff0d87cd47071d9f19e99c5eebcc52a5c2f483ad3b89726fc3fcfe7bad92fc7)
BibTeX
@misc{dc98223ee5171e,
title = {EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing — arXiv:2609.17953},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.17953},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_98223ee5171e; sha256 7ff0d87cd47071d9f19e99c5eebcc52a5c2f483ad3b89726fc3fcfe7bad92fc7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17953. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Vision-Language Models (VLMs) can produce Natural Language Explanations (NLEs) that sound plausible yet remain inconsistent with the visual evidence they cite. We present Explanation-Driven Counterfactual Testing (EDCT), an intervention-based protocol that extracts visual concepts cited in a model's explanation, applies verified minimal edits to them, and tests whether the resulting answer and explanation remain consistent with the edited image. Using this protocol, we create EDCT-Bench, a comprehensive benchmark spanning three complementary domains: knowledge-intensive visual question answering (OK-VQA), safety-critical driving (DriveLM), and 3D spatial reasoning (3DSRBench). Across the evaluated VLMs, EDCT reveals substantial faithfulness gaps, with models frequently producing responses inconsistent with verified visual changes. Finally, our fine-tuning study suggests that EDCT-generated counterfactuals provide high-impact training signals. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.17953 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T15:45:19.737Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).