Metraipolis › archive › dc_da43acdd5fc1

Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation — arXiv:2609.02998

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.02998
License (as archived)
CC0-1.0
Archived
2026-10-17T01:11:44.807Z
SHA-256 of the archived text
3ce287601b4df0b116c5abe48577a8a5dabf96e03709f4ff3ed2d8d88de4fcc1
Stable URL of this page
https://metraipolis.com/documento/dc_da43acdd5fc1

How to cite

Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation — arXiv:2609.02998 — arXiv (en), https://arxiv.org/abs/2609.02998 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_da43acdd5fc1 (Metraipolis archive dc_da43acdd5fc1, sha256 3ce287601b4df0b116c5abe48577a8a5dabf96e03709f4ff3ed2d8d88de4fcc1)
BibTeX
@misc{dcda43acdd5fc1,
  title = {Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation — arXiv:2609.02998},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.02998},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_da43acdd5fc1; sha256 3ce287601b4df0b116c5abe48577a8a5dabf96e03709f4ff3ed2d8d88de4fcc1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02998. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-policy distillation (OPD) accelerates post-training by providing dense token-level supervision from a frozen teacher on the student's own rollouts. Vanilla OPD applies this supervision uniformly across prompts, without checking whether the teacher is reliable for each prompt. Because reverse KL is mode-seeking, a confidently wrong teacher can induce a strong yet misleading update. Distributional proxies, such as entropy or teacher-student likelihood agreement, measure uncertainty or agreement but do not directly verify outcome correctness. We introduce Teacher-Gated On-Policy Distillation (TGOPD), built on the principle that teacher reliability should be verified at the prompt level before dense supervision is admitted. TGOPD estimates reliability from a small set of verifier-scored teacher probes and routes each prompt exclusively to dense OPD when the reliability check passes or to verifier-grounded GRPO otherwise. Across 4B and 35B students in mathematics, code, and instruction following, TGOPD outperforms Vanilla OPD in all six single-domain settings and achieves higher seven-benchmark averages at both scales under multi-domain training. By using otherwise-idle teacher capacity for reliability estimation, TGOPD also reduces teacher-side compute waste in asynchronous OPD, increasing teacher-node GPU utilization from 9.8% to 78.9% in the measured 4B single-domain run.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-04 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.02998 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T00:28:02.940Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-04?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).