Metraipolis › archive › dc_364fe04c585f
Know Thyself, Teach Thyself: Internal Information Flow for Selective Self-Distillation — arXiv:2609.36695
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36695
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T17:29:22.114Z
- SHA-256 of the archived text
5de41613ba6fe4bef2deb9bb1841bcb462d208851e46fa906c975fa2970e0a0b- Stable URL of this page
https://metraipolis.com/documento/dc_364fe04c585f
How to cite
Know Thyself, Teach Thyself: Internal Information Flow for Selective Self-Distillation — arXiv:2609.36695 — arXiv (en), https://arxiv.org/abs/2609.36695 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_364fe04c585f (Metraipolis archive dc_364fe04c585f, sha256 5de41613ba6fe4bef2deb9bb1841bcb462d208851e46fa906c975fa2970e0a0b)
BibTeX
@misc{dc364fe04c585f,
title = {Know Thyself, Teach Thyself: Internal Information Flow for Selective Self-Distillation — arXiv:2609.36695},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36695},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_364fe04c585f; sha256 5de41613ba6fe4bef2deb9bb1841bcb462d208851e46fa906c975fa2970e0a0b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36695. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Self-distillation turns knowledge distillation into a closed learning loop and offers a path toward recursive self-improvement. Without an external teacher, however, the model must determine both what information can improve its supervision and which induced changes should be learned. Existing methods typically improve teacher-generated data or select training examples in isolation, leaving the information transferred between these stages unmeasured. We introduce InFlow, a retrieval-guided on-policy self-distillation framework that models this process as potential-to-realized information flow. InFlow first retrieves potentially informative sources using certainty-calibrated hidden-state trajectories, then measures their realized effect through the Jensen--Shannon divergence between the teacher's initial and retrieval-conditioned answer beliefs. Examples with larger belief shifts are selected for on-policy distillation. Our analysis formalizes the information optimized by retrieval and selection and relates the answer-level shift to the teacher--student distillation gap. Across four open-weight language models and three knowledge domains, InFlow achieves the strongest cross-model average among the compared selection methods, with ablations supporting both stages of the framework. Our code is available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.36695 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T16:59:48.219Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).