Metraipolis › archive › dc_aefe83a48be7

Mind What Matters for Reasoning: Aligning Cross-Modal Attention via Selective Probability Mass Concentration — arXiv:2609.29940

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.29940
License (as archived)
CC0-1.0
Archived
2026-10-08T04:27:53.127Z
SHA-256 of the archived text
3bf0f322315169cf18b084326a1b7f8d28da372ee2ff02b95a3ccc790f0934c5
Stable URL of this page
https://metraipolis.com/documento/dc_aefe83a48be7

How to cite

Mind What Matters for Reasoning: Aligning Cross-Modal Attention via Selective Probability Mass Concentration — arXiv:2609.29940 — arXiv (en), https://arxiv.org/abs/2609.29940 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aefe83a48be7 (Metraipolis archive dc_aefe83a48be7, sha256 3bf0f322315169cf18b084326a1b7f8d28da372ee2ff02b95a3ccc790f0934c5)
BibTeX
@misc{dcaefe83a48be7,
  title = {Mind What Matters for Reasoning: Aligning Cross-Modal Attention via Selective Probability Mass Concentration — arXiv:2609.29940},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.29940},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aefe83a48be7; sha256 3bf0f322315169cf18b084326a1b7f8d28da372ee2ff02b95a3ccc790f0934c5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29940. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multimodal large language models (MLLMs) achieve strong performance on visual reasoning tasks, yet remain prone to hallucinations and over-reliance on language priors, often generating answers without adequately using task-relevant visual evidence. Existing approaches primarily improve reasoning through reasoning-oriented supervision or inference-time strategies. In this work, we study a complementary question: can multimodal reasoning be improved by strengthening implicit visual grounding without directly supervising the reasoning process? Motivated by the functional specialization of attention heads, we investigate whether reasoning can be improved by guiding only the heads most responsive to visual evidence grounding. We propose Selective Probability Mass Concentration (sPMC), a training framework that identifies grounding-responsive heads and selectively regularizes their text-to-image attention. sPMC treats normalized attention over visual tokens as a spatial probability distribution and encourages the probability mass to be assigned to semantically relevant regions using segmentation-derived spatial priors. Adaptive Head Selection restricts this guidance to visually responsive heads while leaving the remaining heads unconstrained to preserve their complementary functions. Across 6 multimodal benchmark suites, sPMC achieves an average zero-shot improvement of 3% and gains of up to 11.3% across multiple MLLMs while regularizing only 3%-15% of their attention heads. These results demonstrate that targeted guidance of sparse and implicit visual evidence pathways can directly improve multimodal reasoning.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.29940 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:58.190Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).