Metraipolis › archive › dc_2dc9ff4f96be
Shuffled OPSD: When Semantic Consistency Hurts On-Policy Self-Distillation? — arXiv:2611.04958
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.04958
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T12:30:40.270Z
- SHA-256 of the archived text
8d0d86a9b16b8b8b5ac75dcf4272f0015550862292263574df50da2a0d718dd9- Stable URL of this page
https://metraipolis.com/documento/dc_2dc9ff4f96be
How to cite
Shuffled OPSD: When Semantic Consistency Hurts On-Policy Self-Distillation? — arXiv:2611.04958 — arXiv (en), https://arxiv.org/abs/2611.04958 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2dc9ff4f96be (Metraipolis archive dc_2dc9ff4f96be, sha256 8d0d86a9b16b8b8b5ac75dcf4272f0015550862292263574df50da2a0d718dd9)
BibTeX
@misc{dc2dc9ff4f96be,
title = {Shuffled OPSD: When Semantic Consistency Hurts On-Policy Self-Distillation? — arXiv:2611.04958},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.04958},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2dc9ff4f96be; sha256 8d0d86a9b16b8b8b5ac75dcf4272f0015550862292263574df50da2a0d718dd9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04958. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Knowledge distillation improves a student model's reasoning ability through supervision from a teacher. On-policy self-distillation (OPSD) uses the same model as both teacher and student under different contexts: the student generates reasoning trajectories from the question alone, while the teacher additionally receives a complete worked example and provides token-level supervision along those trajectories. Because the problem statement is shared, its content and ordering shape both student generation and teacher feedback. Preserving its semantic structure therefore seems natural, but does it always improve learning? We introduce Shuffled OPSD, which shuffles the words of the shared problem statement while keeping the teacher-only worked example intact. Across three mathematical reasoning benchmarks and three Qwen3 model scales, the method improves mean accuracy over both an intact-target control and Standard OPSD under matched training configurations. The largest gain over Standard OPSD is 3.52 percentage points on Qwen3-1.7B. Further controls show that preserving larger word blocks offers no advantage over word-level shuffling at the evaluated scales, whereas shuffling the teacher-only worked example reduces accuracy at every scale. Cross-problem mixing and target-removal experiments further distinguish the roles of problem content, word order, and teacher context. These results show that disrupting the shared problem statement can improve learning, while disrupting the teacher-only reference hurts performance. They challenge semantic coherence as a universally beneficial principle for OPSD and motivate closer study of how input structure shapes student generation and teacher supervision. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.04958 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T12:26:47.304Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).