Metraipolis › archive › dc_cdd4d0872dd6
WOVEN: Weaving Visual World Modeling into Multimodal LLMs — arXiv:2610.12417
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.12417
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:13:54.633Z
- SHA-256 of the archived text
74ade5e5481d53719fc2fa1b81cf8bd6a6dbd81d6cd6519b83f38df7d2c6ec30- Stable URL of this page
https://metraipolis.com/documento/dc_cdd4d0872dd6
How to cite
WOVEN: Weaving Visual World Modeling into Multimodal LLMs — arXiv:2610.12417 — arXiv (en), https://arxiv.org/abs/2610.12417 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cdd4d0872dd6 (Metraipolis archive dc_cdd4d0872dd6, sha256 74ade5e5481d53719fc2fa1b81cf8bd6a6dbd81d6cd6519b83f38df7d2c6ec30)
BibTeX
@misc{dccdd4d0872dd6,
title = {WOVEN: Weaving Visual World Modeling into Multimodal LLMs — arXiv:2610.12417},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.12417},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cdd4d0872dd6; sha256 74ade5e5481d53719fc2fa1b81cf8bd6a6dbd81d6cd6519b83f38df7d2c6ec30},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.12417. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning. We hypothesize that these failures reflect a shared deficit in visual transition reasoning, and test whether this capability can serve as a shared training primitive, one that different models can learn from different supervision sources and reuse across different tasks, with a systematic training recipe. Existing benchmarks document these deficits separately but do not support controlled comparisons across scenes, actions, and reasoning operations. We therefore introduce WOVEN, a training source and benchmark for visual transition reasoning that organizes transition supervision by scene, action, and reasoning type, using diverse, realistic rollouts from video-pretrained generative models: 36,076 examples across 20 scene types, 5 action types, and 8 reasoning types. We first evaluate 38 frontier MLLMs (e.g., GPT-5.4 and Qwen3-VL-235B-A22B) and find a substantial and systematic deficit: even the strongest models fall far below humans, and the failures recur across model families and persist with scale. We then train MLLMs at multiple scales on WOVEN and find that they learn a shared capability that transfers broadly: training subsets of only about 2,000 items each collectively improve 22 of 26 external benchmarks by up to 27.3 percentage points, and WOVEN data can replace 30-50% of a task's own training data with comparable accuracy. Controlled comparisons further yield a training recipe for visual world modeling, validated prospectively on held-out benchmarks: select supervision by the reasoning operation it teaches rather than by the actions, scenes, or domains it shows, and prefer larger changes to the visual state for robustness. Our work establishes visual transition reasoning as a reusable foundation for systematic visual world-model training in MLLMs. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Computation and Language (cs.CL); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.12417 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T05:47:15.592Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).