Metraipolis › archive › dc_49f375f0e1eb

Mental Imagery That Matters: Curing Latent Collapse in Multimodal Reasoning — arXiv:2611.10423

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.10423
License (as archived)
CC0-1.0
Archived
2026-11-10T18:42:28.604Z
SHA-256 of the archived text
8a7851a41c2e24f2c85913a5bc86c0de13e6068908f666cbf860299a73a4f5f8
Stable URL of this page
https://metraipolis.com/documento/dc_49f375f0e1eb

How to cite

Mental Imagery That Matters: Curing Latent Collapse in Multimodal Reasoning — arXiv:2611.10423 — arXiv (en), https://arxiv.org/abs/2611.10423 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_49f375f0e1eb (Metraipolis archive dc_49f375f0e1eb, sha256 8a7851a41c2e24f2c85913a5bc86c0de13e6068908f666cbf860299a73a4f5f8)
BibTeX
@misc{dc49f375f0e1eb,
  title = {Mental Imagery That Matters: Curing Latent Collapse in Multimodal Reasoning — arXiv:2611.10423},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.10423},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_49f375f0e1eb; sha256 8a7851a41c2e24f2c85913a5bc86c0de13e6068908f666cbf860299a73a4f5f8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.10423. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Mental simulation, the ability to reason by constructing and updating internal visual states, is central to flexible human cognition, but remains elusive in current multimodal large language models. Latent multimodal reasoning offers a promising route toward this capability by interleaving text with continuous visual latent blocks, allowing models to maintain internal visual workspaces without decoding them into pixels. In this paper, we ask whether these latent visual states actually matter. Through a direct perturbation analysis across representative latent-reasoning MLLMs, we find that they often do not: dropping, replacing, or shuffling generated visual latents has only minor effects on the model's own final-answer probability, while equivalent perturbations to the linguistic reasoning trace are substantially more damaging. We call this failure mode Mental Simulation Collapse (MSC): the model appears to perform interleaved visual reasoning, but routes most answer-relevant computation through text. We further diagnose that MSC follows naturally from existing training objectives: visual alignment can make generated latents resemble helper-image embeddings, but it does not require the decoder to rely on them; subsequent text-only supervision further leaves the semantic role of the latent pathway unidentifiable. To address this, we introduce AnchorSim (Anchored Mental Simulation), a contrastive distillation framework that trains models to make their generated visual latents consequential. Across multiple visual reasoning benchmarks, AnchorSim improves performance, while diagnostic re-evaluation shows that visual perturbations now meaningfully affect predictions. Our results expose a fundamental failure mode of latent multimodal reasoning and demonstrate that mental simulation must be anchored to model behavior in order to matter.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.10423 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.587Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).