Metraipolis › archive › dc_fa21e282de33
PIVOT: Pivot-Aware On Policy Self Distillation for Multi-Turn VLM Agents — arXiv:2609.35303
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.35303
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T23:52:42.318Z
- SHA-256 of the archived text
97446182a54cb36e1d06bba16df34a36d20bb64fee0df6741517a56ed401f3c8- Stable URL of this page
https://metraipolis.com/documento/dc_fa21e282de33
How to cite
PIVOT: Pivot-Aware On Policy Self Distillation for Multi-Turn VLM Agents — arXiv:2609.35303 — arXiv (en), https://arxiv.org/abs/2609.35303 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fa21e282de33 (Metraipolis archive dc_fa21e282de33, sha256 97446182a54cb36e1d06bba16df34a36d20bb64fee0df6741517a56ed401f3c8)
BibTeX
@misc{dcfa21e282de33,
title = {PIVOT: Pivot-Aware On Policy Self Distillation for Multi-Turn VLM Agents — arXiv:2609.35303},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.35303},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fa21e282de33; sha256 97446182a54cb36e1d06bba16df34a36d20bb64fee0df6741517a56ed401f3c8},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35303. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reinforcement learning with verifiable rewards (RLVR) via Group-Relative Policy Optimization (GRPO) is widely used for multi-turn VLM agent training, yet it suffers from zero-gradient silence on uniform failures and coarse episode-level credit assignment. While On-Policy Distillation (OPD) and On-Policy Self-Distillation (OPSD) mitigate sparse rewards using hindsight information, their underlying mechanisms remain poorly understood. Through controlled counterfactual rollback probes across five multi-turn VLM agent benchmarks, we reveal that performance gains in OPSD/OPD are largely driven by physical state rollback at the pivot step, defined as the first unrecoverable action without remaining step budget. However, physical state rollbacks are computationally prohibitive and infeasible in real-world environments. To bridge this gap, we present Pivot-Aware Internalized Visual On-Policy Training (PIVOT), an RL framework that internalizes pivot localization and state restoration directly into token-level parameter updates, eliminating environment rollbacks during RL training and additional skill hints at test time. PIVOT unifies three functional roles within a single architecture: a failure Analyzer non-invasively localizes the pivot step and diagnoses failure modes from visual trajectory collages and action logs; a detached Teacher re-scores failed tokens under this privileged diagnostic context; and a Student optimizes joint GRPO and confidence-gated OPD objectives. At test time, both Teacher and Analyzer branches are stripped. Evaluated on five multi-turn VLM agent tasks across cognitive grid puzzles, 3D embodied control and navigation, and generative reasoning, PIVOT achieves 0.90 overall accuracy on Qwen2.5-VL-3B (+8% over SFT+GRPO baseline and +5% over previous SOTA) and scales to 0.92 on Qwen3-VL-2B (+12% over SFT+GRPO baseline). Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.35303 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T23:29:27.548Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).