Metraipolis › archive › dc_94b2d623ab5a
Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts — arXiv:2610.10460
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.10460
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T11:12:09.957Z
- SHA-256 of the archived text
3e6fc0e5c8faa2a3b597bf819059ff29ab2641b7678ba3c39844e862c31920f7- Stable URL of this page
https://metraipolis.com/documento/dc_94b2d623ab5a
How to cite
Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts — arXiv:2610.10460 — arXiv (en), https://arxiv.org/abs/2610.10460 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_94b2d623ab5a (Metraipolis archive dc_94b2d623ab5a, sha256 3e6fc0e5c8faa2a3b597bf819059ff29ab2641b7678ba3c39844e862c31920f7)
BibTeX
@misc{dc94b2d623ab5a,
title = {Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts — arXiv:2610.10460},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.10460},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_94b2d623ab5a; sha256 3e6fc0e5c8faa2a3b597bf819059ff29ab2641b7678ba3c39844e862c31920f7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.10460. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multi-teacher on-policy distillation (MOPD) is used in two settings. In common-domain composition, several teachers score each student rollout from one prompt domain and their signals form a single target; in routed-domain distillation, prompts from different domains are assigned to the corresponding specialist. Both settings usually transfer each teacher's endpoint policy, which mixes what post-training changed with preferences inherited from the teacher's base. We introduce $\Delta$-MOPD, which transfers each teacher's teacher-minus-base logit shift re-anchored at the student's frozen initialization, and compare it with endpoint supervision in both settings while holding teacher selection fixed. We first expose the mechanism that impedes endpoint transfer: inherited base pull can exceed the post-training shift. Removing it reduces the teacher-term norm ratio and target--student KL. Across our experiments, the results suggest that shift targets are particularly useful when teacher signals are combined at a state. With three composed teachers, $\Delta$-MOPD exceeds endpoint composition by $4.11$ Math and $1.95$ five-benchmark points; with two, it matches endpoint accuracy. Under phased routing, it achieves higher mean performance in both phase orders and reduces the observed order gap from $10.50$ to $6.42$ points. Under interleaved routing, where each update involves one teacher, the two targets perform comparably. The phased results provide supporting evidence that the benefit may extend to signals accumulated across training phases. Target construction is thus an independent design axis in MOPD, complementary to teacher selection. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.10460 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T10:14:06.185Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).