Metraipolis › archive › dc_dddc2413e0fc
MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation — arXiv:2609.30837
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.30837
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T01:39:31.943Z
- SHA-256 of the archived text
472dc443ebf170587a62627a8977b356687888107f27cb7eb58b3cf1d42948bf- Stable URL of this page
https://metraipolis.com/documento/dc_dddc2413e0fc
How to cite
MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation — arXiv:2609.30837 — arXiv (en), https://arxiv.org/abs/2609.30837 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dddc2413e0fc (Metraipolis archive dc_dddc2413e0fc, sha256 472dc443ebf170587a62627a8977b356687888107f27cb7eb58b3cf1d42948bf)
BibTeX
@misc{dcdddc2413e0fc,
title = {MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation — arXiv:2609.30837},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.30837},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dddc2413e0fc; sha256 472dc443ebf170587a62627a8977b356687888107f27cb7eb58b3cf1d42948bf},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30837. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multi-teacher on-policy distillation (MOPD) integrates specialized capabilities into a single student, but existing practice typically hard-routes each prompt to a domain-matched teacher for the entire rollout. This dependence on prompt-level domain labels restricts using unlabeled training mixtures and leaves complementary signals from other teachers unused. We introduce MOPD-Router, a framework that routes supervision over the full teacher pool at each token, without domain labels or training a separate routing model. Its plug-in interface supports different metrics for selecting and weighting teacher-specific OPD signals. Within this interface, we propose ExpertAlign, which scores each teacher by whether its correction to the student at the current token expresses the specialization that teacher acquired during post-training, and compare it against two reference metrics built on teacher confidence (Entropy) and teacher-student discrepancy (Novelty). Experiments on unlabeled and domain-labeled training mixtures under strong-to-weak and same-size distillation scenarios show that ExpertAlign achieves the strongest overall performance in all four settings. On unlabeled data, it improves the overall score by 5.88 (+12.3%) points over Mean aggregation; on domain-labeled data, it outperforms standard MOPD by 3.95 (+7.8%) points without using available domain labels. These results demonstrate token-level routing can exploit cross-domain complementary supervision, and reduce exclusive reliance on prompt-level domain assignment. Code is available at: this https URL . Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.30837 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T01:12:14.488Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).