Metraipolis › archive › dc_bcfb355cb01b
OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories — arXiv:2608.08557
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.08557
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:50:04.594Z
- SHA-256 of the archived text
d8aeaad044a229362414a657ef8f8363598b334bc020f6bc0ea8b1b8a0e2f032- Stable URL of this page
https://metraipolis.com/documento/dc_bcfb355cb01b
How to cite
OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories — arXiv:2608.08557 — arXiv (en), https://arxiv.org/abs/2608.08557 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bcfb355cb01b (Metraipolis archive dc_bcfb355cb01b, sha256 d8aeaad044a229362414a657ef8f8363598b334bc020f6bc0ea8b1b8a0e2f032)
BibTeX
@misc{dcbcfb355cb01b,
title = {OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories — arXiv:2608.08557},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.08557},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bcfb355cb01b; sha256 d8aeaad044a229362414a657ef8f8363598b334bc020f6bc0ea8b1b8a0e2f032},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08557. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Visual tool use has emerged as a fundamental capability for multimodal agents to actively acquire evidence beyond a fixed image encoding. The prevailing recipe learns this capability from teacher-generated trajectories filtered for answer correctness, implicitly assuming that every successful demonstration provides effective supervision. We argue this assumption is flawed: a strong teacher often reaches the correct answer without needing its tool calls, and imitating such trajectories teaches a student that tool calls accompany correct answers, not that tool observations ground them. We present OpenVisTool, an open framework for constructing instructive visual tool-use trajectories that provide effective supervision for tool learning. The key insight is that a trajectory should be retained only if its answer is correct (outcome validity) and its tool observations causally contribute to that answer (causal utility). The framework operates in three stages: difficulty screening to select queries that are not reliably answerable without tools, domain-specific trajectory synthesis to elicit coherent tool-use trajectories, and supervision verification to jointly test both conditions. Rather than encouraging models to imitate tool calls, the resulting supervision teaches when and how visual evidence should be acquired. Using this framework, we construct OpenVisTool-42K, a dataset spanning five visual reasoning domains, together with OpenVisTool-Bench, a benchmark covering the same domains. Across four backbones (4B-27B), fine-tuning on OpenVisTool-42K consistently improves visual tool-use performance and yields gains on two out-of-distribution benchmarks; the larger models approach leading closed-source systems. The evidence suggests that effective visual tool use is learned from causally grounded supervision rather than tool-calling patterns. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.08557 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:49:41.862Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).