Metraipolis › archive › dc_3069a75a355e
Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation — arXiv:2610.21234
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.21234
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T10:20:14.571Z
- SHA-256 of the archived text
e73af56c446979df46b4b673c3f491d3e57c1b64da394a961ce61ea690ad28d7- Stable URL of this page
https://metraipolis.com/documento/dc_3069a75a355e
How to cite
Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation — arXiv:2610.21234 — arXiv (en), https://arxiv.org/abs/2610.21234 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3069a75a355e (Metraipolis archive dc_3069a75a355e, sha256 e73af56c446979df46b4b673c3f491d3e57c1b64da394a961ce61ea690ad28d7)
BibTeX
@misc{dc3069a75a355e,
title = {Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation — arXiv:2610.21234},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.21234},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3069a75a355e; sha256 e73af56c446979df46b4b673c3f491d3e57c1b64da394a961ce61ea690ad28d7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21234. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
On-policy distillation (OPD) has recently been introduced as a promising paradigm for continual learning, where a student policy can be supervised at the token-level. Typically, OPD is performed using a single teacher per prompt rollout (or even per training run). However, this ignores a whole other axis of flexibility that is available due to the on-policy nature - token-level teacher selection during rollouts. Building on this insight, we theoretically explore what makes a teacher and a student compatible for OPD, and leverage our findings to derive several empirical heuristics for teacher selection. In particular, our framework captures several desiderata for OPD: compatible teacher-student reasoning patterns, teacher competence, and induced student diversity. Theoretically, we categorize several types of teacher routing schemes, and we prove that under different scenarios, different routing schemes are likely to be beneficial for optimization and student diversity. We back up our findings with extensive experimental results in domains such as math, coding, kernel generation, and visual reasoning, where we find upwards of 10 percentage points of improvement on pass@$k$ when compared to vanilla OPD and the teacher experts on certain tasks, indicating both capability improvement and diversity over high-quality responses. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.21234 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:48:02.696Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).