Metraipolis › archive › dc_d4e9d4b747b0

Beyond Prediction: Steering VLM Agents with Retrospective World Modeling — arXiv:2609.39101

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.39101
License (as archived)
CC0-1.0
Archived
2026-10-07T12:42:28.259Z
SHA-256 of the archived text
67c089ef3004bfb3aaa921f13ee4041f5a6b7a3727e3e3eebffa6fe8977f491c
Stable URL of this page
https://metraipolis.com/documento/dc_d4e9d4b747b0

How to cite

Beyond Prediction: Steering VLM Agents with Retrospective World Modeling — arXiv:2609.39101 — arXiv (en), https://arxiv.org/abs/2609.39101 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d4e9d4b747b0 (Metraipolis archive dc_d4e9d4b747b0, sha256 67c089ef3004bfb3aaa921f13ee4041f5a6b7a3727e3e3eebffa6fe8977f491c)
BibTeX
@misc{dcd4e9d4b747b0,
  title = {Beyond Prediction: Steering VLM Agents with Retrospective World Modeling — arXiv:2609.39101},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.39101},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d4e9d4b747b0; sha256 67c089ef3004bfb3aaa921f13ee4041f5a6b7a3727e3e3eebffa6fe8977f491c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39101. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Equipping VLM agents with world modeling capabilities has shown strong potential for complex reasoning and long-horizon planning, while reducing the dependence of policy learning on costly real-world interactions. Existing methods mainly rely on prospective simulation to predict the consequences of candidate actions. However, this forward-only paradigm focuses on what will happen next and provides limited constraints for verifying whether an action is causally consistent with the observed state transition, which can lead to plausible-looking but physically incoherent behaviors. In this paper, we challenge the view of world modeling as only prospective prediction and introduce Retrospective World Modeling, a new agent learning paradigm that enables agents to reason backward by estimating the retrospective attribution distribution $P(\hat{a}{t}|s_t, s{t+1})$ for the action that most likely caused a given transition. Based on this capability, we formulate the Self-Consistency Reward (SCR), an intrinsic signal that measures the probabilistic consistency between the policy action and the retrospective explanation. Integrating SCR into reinforcement learning provides dense transition-level feedback and steers agents toward behaviors that are both task-effective and physically grounded. Extensive experiments across diverse agentic tasks show that our method substantially improves policy robustness and generalization over prospective-only world modeling baselines.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.39101 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:13.730Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).