Metraipolis › archive › dc_8aec399b5052

MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation — arXiv:2610.11989

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.11989
License (as archived)
CC0-1.0
Archived
2026-10-09T05:48:28.868Z
SHA-256 of the archived text
c737736aa8d4cd098e61220a5bdb1a9d6ee9fb93020f1d3f2ca6539994d343ff
Stable URL of this page
https://metraipolis.com/documento/dc_8aec399b5052

How to cite

MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation — arXiv:2610.11989 — arXiv (en), https://arxiv.org/abs/2610.11989 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8aec399b5052 (Metraipolis archive dc_8aec399b5052, sha256 c737736aa8d4cd098e61220a5bdb1a9d6ee9fb93020f1d3f2ca6539994d343ff)
BibTeX
@misc{dc8aec399b5052,
  title = {MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation — arXiv:2610.11989},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.11989},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8aec399b5052; sha256 c737736aa8d4cd098e61220a5bdb1a9d6ee9fb93020f1d3f2ca6539994d343ff},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.11989. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-policy distillation (OPD) trains a student on its own generated responses using token-level teacher supervision. However, uniform weighting overlooks differences in token learning value, while existing weighting methods rely on predefined mappings from prediction signals to token weights. These mappings are not learned from the effectiveness of the resulting student updates, limiting their ability to adapt to evolving learning needs. In this paper, we propose MetaOPD, a bilevel optimization framework that jointly learns the student model and a lightweight token-weighting network. The inner objective updates the student through weighted OPD, while the outer objective optimizes the weighting network using validation loss on reference solutions after a virtual student update. Differentiating through this update connects weighting decisions to their effects on post-update performance, allowing the mapping from prediction signals to token weights to evolve alongside the student. Experiments on six mathematical reasoning and three out-of-domain datasets, covering two student scales and seven baselines, demonstrate the effectiveness of MetaOPD, with Avg@8/Pass@8 gains over OPD of 1.99/5.97 percentage points for the 0.6B student and 2.25/6.41 points for the 1.7B student.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.11989 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:47:00.992Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).