Metraipolis › archive › dc_dd45817a2b5c
Beyond Imitation: Auditing the Recoverability of Reasoning in Distilled Models — arXiv:2609.26216
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.26216
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T15:07:50.816Z
- SHA-256 of the archived text
8ed4ac7fb8e26f2c82ca7e3a3bef96455d1a6c0adbb8367685bb256acb72ee85- Stable URL of this page
https://metraipolis.com/documento/dc_dd45817a2b5c
How to cite
Beyond Imitation: Auditing the Recoverability of Reasoning in Distilled Models — arXiv:2609.26216 — arXiv (en), https://arxiv.org/abs/2609.26216 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dd45817a2b5c (Metraipolis archive dc_dd45817a2b5c, sha256 8ed4ac7fb8e26f2c82ca7e3a3bef96455d1a6c0adbb8367685bb256acb72ee85)
BibTeX
@misc{dcdd45817a2b5c,
title = {Beyond Imitation: Auditing the Recoverability of Reasoning in Distilled Models — arXiv:2609.26216},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.26216},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dd45817a2b5c; sha256 8ed4ac7fb8e26f2c82ca7e3a3bef96455d1a6c0adbb8367685bb256acb72ee85},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26216. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
A correct teacher solution becomes useful supervision when the receiving student can continue its reasoning. We measure this compatibility with prefix recovery: after revealing 25%, 50%, or 75% of a verified solution, we test whether the student completes it correctly. We connect recovery to the cosine conflict between cross-entropy and reverse-KL gradients over the full vocabulary. Across adjacent Qwen3 teacher-student pairs from 0.6B to 8B parameters, reverse-KL distillation delivers its most consistent mathematical and code improvements for the two students below 2B parameters. On a fixed cohort of 1,000 trajectories, average prefix recovery rises from 71.0% to 91.9% as student size increases from 0.6B to 4B, and the robust-fragile recovery gap contracts from 46.0 to 14.4 percentage points. With the teacher fixed at 8B, conflict separation falls from 0.993 to 0.233. An independent objective intervention finds the largest reverse-KL rescue on fragile trajectories. The three measurements locate the same capacity-dependent transfer regime: distribution matching has the greatest headroom when correct traces remain unevenly recoverable. Prefix recovery provides a practical diagnostic for selecting costly distribution-level distillation. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.26216 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T14:37:29.468Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).