Metraipolis › archive › dc_09bacc587162
Learning Where to Look: Observation Policy Optimization for Thinking with Images — arXiv:2611.04943
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.04943
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T19:04:44.781Z
- SHA-256 of the archived text
90e033b328313d78bfce5d2948437eb65b96a501be4e4ab371da47b3ca1d0d8e- Stable URL of this page
https://metraipolis.com/documento/dc_09bacc587162
How to cite
Learning Where to Look: Observation Policy Optimization for Thinking with Images — arXiv:2611.04943 — arXiv (en), https://arxiv.org/abs/2611.04943 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_09bacc587162 (Metraipolis archive dc_09bacc587162, sha256 90e033b328313d78bfce5d2948437eb65b96a501be4e4ab371da47b3ca1d0d8e)
BibTeX
@misc{dc09bacc587162,
title = {Learning Where to Look: Observation Policy Optimization for Thinking with Images — arXiv:2611.04943},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.04943},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_09bacc587162; sha256 90e033b328313d78bfce5d2948437eb65b96a501be4e4ab371da47b3ca1d0d8e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04943. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Complex visual question answering often requires vision-language models to think with images, i.e., actively acquire local visual evidence by zooming into informative regions of high-resolution images. Our error analysis shows that many failures in this setting originate from incorrect observation actions, where the model fails to localize key regions or obtain crops that contain sufficient evidence for answering. We further observe that bounding-box tokens exhibit substantially higher generation entropy than ordinary language tokens, indicating high uncertainty when the model decides where to look. However, existing reinforcement learning methods based on final-answer rewards provide only trajectory-level feedback, making it difficult to assign precise credit to the observation actions. We propose Observation Policy Optimization (OPO), an RL algorithm for optimizing observation actions in tool-augmented thinking with images. OPO consists of two core modules. First, Uncertainty-Guided Observation Branching (UOB) treats each image zoom-in tool call as an observation event and selectively branches at high-uncertainty, low-redundancy events during rollout, enabling efficient exploration of alternative cropping decisions and their resulting visual evidence. Second, Evidence-Localized Advantage Attribution (ELAA) compares sibling branches using local evidence-sufficiency rewards and assigns credit only to the corresponding observation-action tokens. By decoupling observation-action optimization from trajectory-level outcome supervision, OPO improves visual evidence acquisition and further enhances visual reasoning performance. Experiments show that OPO consistently improves Qwen3-VL models across scales and achieves competitive performance against larger models and tool-augmented baselines. Code is publicly available. Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2611.04943 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T18:39:37.552Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).