Metraipolis › archive › dc_31fd80e4aedd

Learning from Teacher Continuations at Student States — arXiv:2609.36246

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.36246
License (as archived)
CC0-1.0
Archived
2026-10-07T16:56:28.889Z
SHA-256 of the archived text
9ef4549e93f831ede7f3fa5ae76271ff2fb54eedd3c611d3ee8e25287b4a064d
Stable URL of this page
https://metraipolis.com/documento/dc_31fd80e4aedd

How to cite

Learning from Teacher Continuations at Student States — arXiv:2609.36246 — arXiv (en), https://arxiv.org/abs/2609.36246 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_31fd80e4aedd (Metraipolis archive dc_31fd80e4aedd, sha256 9ef4549e93f831ede7f3fa5ae76271ff2fb54eedd3c611d3ee8e25287b4a064d)
BibTeX
@misc{dc31fd80e4aedd,
  title = {Learning from Teacher Continuations at Student States — arXiv:2609.36246},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.36246},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_31fd80e4aedd; sha256 9ef4549e93f831ede7f3fa5ae76271ff2fb54eedd3c611d3ee8e25287b4a064d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36246. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We present OLIVE (OnLine InterVEntion). At each iteration, the evolving student policy generates a new prefix, the teacher continues it autoregressively, and the student is updated using cross-entropy computed on the teacher-generated tokens. Each design choice targets a corresponding limitation of existing distillation methods: (1) sequential covariate shift in offline supervised fine-tuning (SFT) on fixed teacher trajectories, (2) fragmented supervision under prefix failure in token-level on-policy distillation (OPD), and (3) the need for access to teacher token probabilities in distribution-matching distillation. OLIVE achieves higher reasoning performance than OPD (with a top-16 KL approximation) at comparable GPU-hour cost. Our asynchronous implementation further reduces OLIVE's total training time by 23.8\%. We evaluate OLIVE on both hard reasoning tasks and agentic tasks which reflects modern post-training scenarios, and it consistently outperforms existing distillation methods under the same training budget. By regenerating prefixes from the evolving student, OLIVE continues improving after offline distillation plateaus while better preserving the general capabilities and plasticity of the student. Using only text from GPT-5.4-mini, continuously training with OLIVE outperforms offline SFT from the same teacher by 13\% on ScienceWorld. These results support OLIVE as an effective and efficient approach to online language-model distillation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.36246 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T16:27:03.195Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).