Metraipolis › archive › dc_6bbc2ae51efd

What Does Privileged Information Add to On-Policy Self-Distillation? — arXiv:2609.20612

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.20612
License (as archived)
CC0-1.0
Archived
2026-10-10T03:48:24.691Z
SHA-256 of the archived text
e0203f3b06ea2f6cfa23ffbc8c4045e8c6d5fa517e9fad33d324b9add471cbe8
Stable URL of this page
https://metraipolis.com/documento/dc_6bbc2ae51efd

How to cite

What Does Privileged Information Add to On-Policy Self-Distillation? — arXiv:2609.20612 — arXiv (en), https://arxiv.org/abs/2609.20612 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6bbc2ae51efd (Metraipolis archive dc_6bbc2ae51efd, sha256 e0203f3b06ea2f6cfa23ffbc8c4045e8c6d5fa517e9fad33d324b9add471cbe8)
BibTeX
@misc{dc6bbc2ae51efd,
  title = {What Does Privileged Information Add to On-Policy Self-Distillation? — arXiv:2609.20612},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.20612},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6bbc2ae51efd; sha256 e0203f3b06ea2f6cfa23ffbc8c4045e8c6d5fa517e9fad33d324b9add471cbe8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.20612. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-policy self-distillation (OPSD) lets a language model learn from a frozen copy of itself that sees an answer or a worked solution. Giving the teacher this extra information seems to offer the student more to learn, but how much does it add beyond distillation itself? To isolate that contribution, we construct AMPLE-Math, a reusable suite of 5,319 mathematical problems with six reasoning views that share the same answer, and compare each view with matched reference-free distillation. With a thinking-enabled teacher supervising direct-response rollouts, reference-free distillation accounts for much of Qwen3-1.7B's improvement under thinking-enabled evaluation, both in domain and on external benchmarks. Evidence for an additional reference benefit is modest in Qwen, strongest for a polished solution, whereas complete traces add two percentage points in SmolLM3-3B at step 50. These benefits depend on the student being trained. At the same checkpoint, replacing short direct-response rollouts with long thinking-enabled rollouts turns gains into losses in both families while the problems, references, and evaluation stay fixed. Teacher profiles and matched loss interventions in Qwen further show that changing token-level supervision can leave student behavior largely unchanged. Together, these findings suggest that OPSD can improve access to existing reasoning capabilities through parameters shared by direct-response and thinking-enabled inference. The value of a privileged reference is what it adds to this cross-mode transfer, not how much of the solution it reveals.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.20612 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T03:24:08.169Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).