Metraipolis › archive › dc_8a19a6beb227

MoGround: Measuring and Mitigating Modality Distraction in Vision-Language Models — arXiv:2609.33431

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33431
License (as archived)
CC0-1.0
Archived
2026-10-07T22:37:26.216Z
SHA-256 of the archived text
08ebc60ff5b99186453445710aec1d970a52aa8a4e2bfdca0e72cb97695af98f
Stable URL of this page
https://metraipolis.com/documento/dc_8a19a6beb227

How to cite

MoGround: Measuring and Mitigating Modality Distraction in Vision-Language Models — arXiv:2609.33431 — arXiv (en), https://arxiv.org/abs/2609.33431 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8a19a6beb227 (Metraipolis archive dc_8a19a6beb227, sha256 08ebc60ff5b99186453445710aec1d970a52aa8a4e2bfdca0e72cb97695af98f)
BibTeX
@misc{dc8a19a6beb227,
  title = {MoGround: Measuring and Mitigating Modality Distraction in Vision-Language Models — arXiv:2609.33431},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33431},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8a19a6beb227; sha256 08ebc60ff5b99186453445710aec1d970a52aa8a4e2bfdca0e72cb97695af98f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33431. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We release MoGround, a vision-language dataset spanning four visual domains in which the answer to every question is guaranteed to be available from exactly one modality. This guarantee enables us to measure modality distraction, the failure in which a model answers a question correctly from one modality alone and then flips to a wrong answer once irrelevant content from the other modality is added. Existing probes rarely establish single-modality answerability this way, making it hard to isolate distraction in the first place. Across seven open-source VLMs, we find that modality distraction is not universal but model-dependent. The weaker-grounded modality is the more distracted one (r = +0.86), and distraction scales inversely with grounding strength (r = -0.90). The single-modality guarantee also enables a mitigation method that needs to distinguish between relevant and irrelevant context. Trained on one split of MoGround alone, a weight-space robustness vector reduces distraction on all seven models by 9% to 51%, at a cost of only 0.1 average points of accuracy on standard multimodal tasks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.33431 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.943Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).