Metraipolis › archive › dc_ce68fc208d20

Teacher Should Think Ahead: Adaptive Continuations for Reliable On-Policy Distillation — arXiv:2609.22254

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.22254
License (as archived)
CC0-1.0
Archived
2026-10-09T07:06:21.291Z
SHA-256 of the archived text
665c46ac60b88df2a0aec5acb9bcdf835a15547745c4efa4746a8b737685c469
Stable URL of this page
https://metraipolis.com/documento/dc_ce68fc208d20

How to cite

Teacher Should Think Ahead: Adaptive Continuations for Reliable On-Policy Distillation — arXiv:2609.22254 — arXiv (en), https://arxiv.org/abs/2609.22254 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ce68fc208d20 (Metraipolis archive dc_ce68fc208d20, sha256 665c46ac60b88df2a0aec5acb9bcdf835a15547745c4efa4746a8b737685c469)
BibTeX
@misc{dcce68fc208d20,
  title = {Teacher Should Think Ahead: Adaptive Continuations for Reliable On-Policy Distillation — arXiv:2609.22254},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.22254},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ce68fc208d20; sha256 665c46ac60b88df2a0aec5acb9bcdf835a15547745c4efa4746a8b737685c469},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22254. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-policy distillation (OPD) is a promising approach for transferring knowledge between language models, where a student receives dense token-level supervision along its own generated trajectories. However, teacher supervision can be unreliable when conditioned on incomplete or low-quality student prefixes. We identify Teacher Uncertainty Contraction (TUC), a systematic phenomenon whereby the teacher's predictive uncertainty decreases as it continues from a student-generated prefix. We theoretically characterize this trade-off through a variance-bias decomposition of teacher-branch gradients, showing that uncertainty contraction reduces variance while teacher-student path divergence increases bias, thereby favoring a finite continuation. Guided by this insight, we propose Adaptive-Continuations On-Policy Distillation (AC-OPD), which augments informative states along student rollouts with teacher continuations and adaptively selects their effective supervision horizons. Experiments on mathematical reasoning and code generation across model scales demonstrate that AC-OPD consistently improves over standard OPD. Controlled-continuations and matched-budget analyses further validate the adaptive-continuations design, highlighting adaptive teacher continuations as an effective principle for reliable on-policy this http URL code will be made publicly available upon publication.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.22254 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:58:09.184Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).