Metraipolis › archive › dc_1caaf635c082
Fisher-Informed Recalibration for Feedback-Based On-Policy Self-Distillation of LLMs — arXiv:2609.34009
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.34009
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T22:37:55.496Z
- SHA-256 of the archived text
31e503376220b08723aab9338bdd4b2c49fc8899e06f2f69c648be7d9d46b53c- Stable URL of this page
https://metraipolis.com/documento/dc_1caaf635c082
How to cite
Fisher-Informed Recalibration for Feedback-Based On-Policy Self-Distillation of LLMs — arXiv:2609.34009 — arXiv (en), https://arxiv.org/abs/2609.34009 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1caaf635c082 (Metraipolis archive dc_1caaf635c082, sha256 31e503376220b08723aab9338bdd4b2c49fc8899e06f2f69c648be7d9d46b53c)
BibTeX
@misc{dc1caaf635c082,
title = {Fisher-Informed Recalibration for Feedback-Based On-Policy Self-Distillation of LLMs — arXiv:2609.34009},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.34009},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1caaf635c082; sha256 31e503376220b08723aab9338bdd4b2c49fc8899e06f2f69c648be7d9d46b53c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34009. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Feedback-based on-policy self-distillation has emerged as a promising approach for enabling foundation models, more specifically Large Language Models (LLMs), to learn from their own outputs under external feedback, with a single model serving as both teacher and student. However, such methods can exhibit unstable optimization, conducive to performance collapse during training. To address this limitation, we propose FIRE (Fisher-Informed REcalibration), a dual-branch framework that recalibrates the supervision applied to correct and incorrect on-policy outputs during fine-tuning. For correct responses, FIRE replaces self-distillation with re-weighted on-policy SFT, while for incorrect ones FIRE identifies feedback components that disproportionately influence the teacher-induced update and recalibrates the feedback-conditioned target accordingly. Both branches are influenced by a token-level radius derived in part from a softmax Fisher trace. FIRE separates which direction feedback should move the model from how far the model should move in that direction, while leaving well-behaved feedback supervision unchanged. Our experiments demonstrate that FIRE provides substantially more stable self-distillation while maintaining strong downstream performance, particularly in settings where standard feedback-conditioned distillation becomes unstable. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.34009 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T22:14:28.087Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).