Metraipolis › archive › dc_7863e401b082
Recovering Reliable Supervision via Critique-Guided On-Policy Distillation — arXiv:2610.20572
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.20572
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T14:20:51.785Z
- SHA-256 of the archived text
d107b132e242597f1e8d9c038cd9db1061b4983e0667c5418a1cb7e3f58cabf9- Stable URL of this page
https://metraipolis.com/documento/dc_7863e401b082
How to cite
Recovering Reliable Supervision via Critique-Guided On-Policy Distillation — arXiv:2610.20572 — arXiv (en), https://arxiv.org/abs/2610.20572 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7863e401b082 (Metraipolis archive dc_7863e401b082, sha256 d107b132e242597f1e8d9c038cd9db1061b4983e0667c5418a1cb7e3f58cabf9)
BibTeX
@misc{dc7863e401b082,
title = {Recovering Reliable Supervision via Critique-Guided On-Policy Distillation — arXiv:2610.20572},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.20572},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7863e401b082; sha256 d107b132e242597f1e8d9c038cd9db1061b4983e0667c5418a1cb7e3f58cabf9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.20572. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
On-policy distillation (OPD) trains a student model on its own rollouts with token-level supervision from a teacher, assuming that the teacher can reach a correct answer from any prefix the student writes. We call this property the recoverability of the teacher and show it fails on incorrect rollouts: the teacher follows student errors, although it recognizes them. We propose CriticOPD, using error recognition to restore recoverability. The teacher critiques an incorrect rollout, allowing the student to rewrite its solution following the error. The resulting output, including the critique as privileged information, is scored by the teacher while the student uses the revision without the critique for training. On two Qwen3 students and six mathematical reasoning benchmarks, CriticOPD improves OPD, with the largest gains on competition problems. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.20572 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T13:53:35.778Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).