Metraipolis › archive › dc_d1c91c015e38
Teacher-Student Gaps Are Not Enough: Outcome-Guided On-Policy Distillation for Multi-Turn Autonomous Agents — arXiv:2609.35319
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.35319
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T21:03:25.838Z
- SHA-256 of the archived text
c1454d3dc893a3b876e81908516e88dbef6bfcff0c8c2a75b3db00b0530379d4- Stable URL of this page
https://metraipolis.com/documento/dc_d1c91c015e38
How to cite
Teacher-Student Gaps Are Not Enough: Outcome-Guided On-Policy Distillation for Multi-Turn Autonomous Agents — arXiv:2609.35319 — arXiv (en), https://arxiv.org/abs/2609.35319 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d1c91c015e38 (Metraipolis archive dc_d1c91c015e38, sha256 c1454d3dc893a3b876e81908516e88dbef6bfcff0c8c2a75b3db00b0530379d4)
BibTeX
@misc{dcd1c91c015e38,
title = {Teacher-Student Gaps Are Not Enough: Outcome-Guided On-Policy Distillation for Multi-Turn Autonomous Agents — arXiv:2609.35319},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.35319},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d1c91c015e38; sha256 c1454d3dc893a3b876e81908516e88dbef6bfcff0c8c2a75b3db00b0530379d4},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35319. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
On-policy distillation (OPD) trains a student on its own trajectories with dense teacher supervision. Recent work on OPD for multi-turn autonomous agents often treats large teacher-student token-level distributional gaps as promising intervention points, linking larger gaps to a greater need for correction. Yet, our empirical analysis reveals a supervision-benefit mismatch: large gaps can be benign, while small gaps can be outcome-critical. Teacher-student gaps capture differences at the current turn, whereas the benefit of teacher guidance depends on how the current student interacts with the environment afterward. The student may still succeed despite choosing an action that differs from the teacher's, while a teacher-preferred action may lead to a state from which the student cannot complete the task. Local gaps alone are therefore not enough to determine whether teacher guidance benefits the current student. Effective supervision should instead emphasize guidance that the current student can translate into better final task outcomes. Accordingly, we propose Outcome-Guided On-Policy Distillation (OG-OPD), which applies trajectory-relative weighting to teacher supervision and calibrates these weights using final task outcomes from paired student continuations. This calibration selectively strengthens supervision on the student's original trajectories at turns where teacher guidance benefits the current student. Across ALFWorld, ScienceWorld, and WebShop, OG-OPD consistently outperforms baselines under diverse settings. It improves task success rates by 3.6-17.7 percentage points over vanilla OPD and by up to 7.0 percentage points over the strongest baseline. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.35319 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:03.313Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).