Metraipolis › archive › dc_8a2cf0d11d65

CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Models — arXiv:2609.38810

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.38810
License (as archived)
CC0-1.0
Archived
2026-10-07T12:44:19.550Z
SHA-256 of the archived text
4fd33367d28c69441b615b0f1a505bc82dd59cfa4bb177461e72da23885254af
Stable URL of this page
https://metraipolis.com/documento/dc_8a2cf0d11d65

How to cite

CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Models — arXiv:2609.38810 — arXiv (en), https://arxiv.org/abs/2609.38810 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8a2cf0d11d65 (Metraipolis archive dc_8a2cf0d11d65, sha256 4fd33367d28c69441b615b0f1a505bc82dd59cfa4bb177461e72da23885254af)
BibTeX
@misc{dc8a2cf0d11d65,
  title = {CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Models — arXiv:2609.38810},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.38810},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8a2cf0d11d65; sha256 4fd33367d28c69441b615b0f1a505bc82dd59cfa4bb177461e72da23885254af},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38810. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As vision language models are increasingly deployed in clinical diagnosis, under standing how they internally resolve competing visual and textual signals becomes a safety imperative. Existing mechanistic analyses remain confined to unimodal text and offer no explanation for why a single misleading sentence can override a correct image based diagnosis, or why a model commits to a confident answer despite insufficient visual evidence. We find that these two safety risks, arbitra tion failure where textual context overrides visual grounding and brake failure where the model commits without adequate evidence, are mediated by spatially disjoint attention head populations: arbitration heads form a mid-to-deep wideband reflecting cross-layer evidence competition, while brake heads concentrate in a narrow middle-to-late layer band that regulates evidence sufficiency and abstention behavior. To ground these observations in causal circuitry, we introduce CRAFT, which localizes each failure mode to a minimal causal head set via dual criteria and verifies necessity and sufficiency through temporal probes and Tuned Lens trajectory analysis. Excising arbitration heads sharply reduces conflict following with negligible degradation on clean inputs, while excising brake heads restores ap propriate abstention under degraded visual evidence. The two interventions target spatially disjoint head sets and produce distinct corrective effects, underscoring the mechanistic separability of the failure modes. Experiments across multiple medical VQA benchmarks and VLM architectures validate both the localization and inter ventions, demonstrating that the identified heads causally drive each failure mode and that targeted modulation generalises without retraining. The code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.38810 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:13.947Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).