Metraipolis › archive › dc_44e85d2d14aa

On Repulsive and Attractive Teachers: Separating Correctness from Behavior in Self-Distillation — arXiv:2609.21561

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.21561
License (as archived): CC0-1.0
Archived: 2026-10-10T01:08:31.724Z
SHA-256 of the archived text: b295a8075e3f94d7b9e7c6601b8c298b7c023c3b6f6cd7b6a7f61aa58adf7640
Stable URL of this page: https://metraipolis.com/documento/dc_44e85d2d14aa

How to cite

On Repulsive and Attractive Teachers: Separating Correctness from Behavior in Self-Distillation — arXiv:2609.21561 — arXiv (en), https://arxiv.org/abs/2609.21561 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_44e85d2d14aa (Metraipolis archive dc_44e85d2d14aa, sha256 b295a8075e3f94d7b9e7c6601b8c298b7c023c3b6f6cd7b6a7f61aa58adf7640)

BibTeX

@misc{dc44e85d2d14aa,
  title = {On Repulsive and Attractive Teachers: Separating Correctness from Behavior in Self-Distillation — arXiv:2609.21561},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.21561},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_44e85d2d14aa; sha256 b295a8075e3f94d7b9e7c6601b8c298b7c023c3b6f6cd7b6a7f61aa58adf7640},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.21561. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-policy self-distillation provides dense, token-level supervision by conditioning a model on privileged information and distilling the resulting teacher distribution back into the model. However, privileged information can change not only what the teacher knows, but also how it behaves, entangling correctness-relevant learning signals with unintended behavioral shifts. We study this effect in reasoning tasks by contrasting attractive self-distillation, which moves the model toward a privileged teacher, with repulsive self-distillation, which moves it away from a privileged teacher. We find that both objectives can induce strong and opposing behavioral shifts: attraction suppresses exploratory reasoning and promotes shorter, more confident responses, whereas repulsion increases response length, can trigger unintended switches into a model's latent thinking mode, and ultimately becomes unstable. Motivated by these observations, we study contrastive self-distillation, which combines attraction toward a correct-solution-conditioned teacher with repulsion from an incorrect-solution-conditioned teacher. In contrast to prior work that combines such distillation signals with a GRPO objective, we isolate the self-distillation objective and study its behavior on its own. We find that the shared behavioral shifts of the two teachers largely cancel, leaving a token-level signal that more directly reflects correctness. Across non-thinking, instruct-only, and already-thinking models, this contrastive objective improves reasoning performance while maintaining stable response lengths.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-21 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.21561 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T00:23:42.545Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).