Metraipolis › archive › dc_d1ca125d8fc6
Learning to Revise Reasoning with Segment-wise On-Policy Distillation — arXiv:2610.02703
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.02703
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-05T22:49:10.650Z
- SHA-256 of the archived text
2370aaa230b3f576d5f1652e2114b814964a2820a78d57aa16054a98d1da5e0e- Stable URL of this page
https://metraipolis.com/documento/dc_d1ca125d8fc6
How to cite
Learning to Revise Reasoning with Segment-wise On-Policy Distillation — arXiv:2610.02703 — arXiv, https://arxiv.org/abs/2610.02703 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d1ca125d8fc6 (Metraipolis archive dc_d1ca125d8fc6, sha256 2370aaa230b3f576d5f1652e2114b814964a2820a78d57aa16054a98d1da5e0e)
BibTeX
@misc{dcd1ca125d8fc6,
title = {Learning to Revise Reasoning with Segment-wise On-Policy Distillation — arXiv:2610.02703},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.02703},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d1ca125d8fc6; sha256 2370aaa230b3f576d5f1652e2114b814964a2820a78d57aa16054a98d1da5e0e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02703. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
On-policy distillation (OPD) improves large language model reasoning by training students on their own rollouts with dense token-wise supervision from the teacher. However, token-wise OPD does not explicitly provide a coherent alternative reasoning step showing how the student's step could be revised to improve subsequent reasoning. Furthermore, this paradigm can become less effective when the student produces a degenerate reasoning prefix, as subsequent teacher supervision remains conditioned on that prefix and may reinforce poor reasoning patterns. In this work, we focus on learning reasoning revision with segment-wise OPD to rework intermediate reasoning steps and better support subsequent reasoning. Through controlled reasoning interventions, we find that replacing student segments with teacher redrafts improves subsequent reasoning accuracy. Therefore, we address the problem of turning teacher redrafts into explicit supervision for learning to revise reasoning. We propose Segment-wise On-Policy Distillation (Seg-OPD), which selects student segments based on an uncertainty metric and obtains corresponding teacher redrafts. Seg-OPD trains the student to prefer teacher redrafts over their paired student segments while retaining dense token-wise OPD supervision. Extensive experiments on mathematical reasoning and competitive programming tasks show that Seg-OPD-trained students achieve higher revision success rates than baselines. Seg-OPD consistently outperforms the compared state-of-the-art baselines in reasoning accuracy with an average relative improvement of 5.22% across diverse models and tasks. Code is available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.02703 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-05T22:45:55.620Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).