Metraipolis › archive › dc_e99134cdd324
Diagnosing the Reference Gap: What Visual Scaffolds Can and Cannot Fix in Multimodal Reasoning — arXiv:2610.39419
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.39419
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T06:17:22.287Z
- SHA-256 of the archived text
0cfcecab3089455c3b85ca7d2b428864588f3e429cc5731c213a4f15b1d32d03- Stable URL of this page
https://metraipolis.com/documento/dc_e99134cdd324
How to cite
Diagnosing the Reference Gap: What Visual Scaffolds Can and Cannot Fix in Multimodal Reasoning — arXiv:2610.39419 — arXiv (en), https://arxiv.org/abs/2610.39419 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e99134cdd324 (Metraipolis archive dc_e99134cdd324, sha256 0cfcecab3089455c3b85ca7d2b428864588f3e429cc5731c213a4f15b1d32d03)
BibTeX
@misc{dce99134cdd324,
title = {Diagnosing the Reference Gap: What Visual Scaffolds Can and Cannot Fix in Multimodal Reasoning — arXiv:2610.39419},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.39419},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e99134cdd324; sha256 0cfcecab3089455c3b85ca7d2b428864588f3e429cc5731c213a4f15b1d32d03},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.39419. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multimodal large language models can recognize visual content yet struggle to reason reliably over it across multiple steps. In counting, a model may see all objects but lose track of which ones it has counted. We argue that such failures, commonly attributed to perception, reveal a reference gap: the failure to bind language to the relevant visual entities, regions, or locations. Motivated by the theory of visual routines, we test whether external references such as numbering can narrow this gap. We introduce ReGap-Bench, a paired benchmark of five visual reasoning tasks in which the scene, question, and answer are fixed and only the reference varies. Results show that task-matched references improve accuracy by up to 25\%, while mismatched references provide little benefit or even hurt performance. These findings identify the reference gap as a bottleneck distinct from visual perception and show that constructing task-appropriate visual references can serve as a training-free visual skill for improving multimodal reasoning. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.39419 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T06:07:23.409Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).