Metraipolis › archive › dc_8a033d6ac4e1

Privileged Context as Drift in On-Policy Self-Distillation — arXiv:2610.07842

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.07842
License (as archived): CC0-1.0
Archived: 2026-10-07T11:41:27.025Z
SHA-256 of the archived text: c47d36f01a717b5dcf860c85416f86c1aedce91ec9ff6f835563b89fc7cdecf8
Stable URL of this page: https://metraipolis.com/documento/dc_8a033d6ac4e1

How to cite

Privileged Context as Drift in On-Policy Self-Distillation — arXiv:2610.07842 — arXiv (en), https://arxiv.org/abs/2610.07842 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8a033d6ac4e1 (Metraipolis archive dc_8a033d6ac4e1, sha256 c47d36f01a717b5dcf860c85416f86c1aedce91ec9ff6f835563b89fc7cdecf8)

BibTeX

@misc{dc8a033d6ac4e1,
  title = {Privileged Context as Drift in On-Policy Self-Distillation — arXiv:2610.07842},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.07842},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8a033d6ac4e1; sha256 c47d36f01a717b5dcf860c85416f86c1aedce91ec9ff6f835563b89fc7cdecf8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.07842. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-policy self-distillation (OPSD) trains a language model to match a copy of itself conditioned on privileged context. Existing work varies what privileged context contains and how it is produced while also changing models, data, and training setups, making the effects of privileged context design difficult to isolate. Motivated by efforts in continual learning to reduce catastrophic forgetting, we study how the choice of privileged context affects policy drift. Specifically, we vary two axes: content (a demonstration, feedback, or rephrase) and source (external, self-generated with a verifier, or self-generated without a verifier). We train Qwen2.5-7B with OPSD across these nine combinations and three datasets, measuring target-task accuracy, prior-task retention, reverse KL from the base policy, and parameter-update geometry. Holding source fixed, changing content spans a wider median KL range than holding content fixed and changing source. The ratio between these ranges is $5.1\times$ for per-token KL and $2.2\times$ for per-sequence KL. Parameter-update geometry shows the same pattern: updates from adapters that share content are more closely aligned (mean cosine $0.571$) than updates from adapters that share source ($0.255$). For continual learning, these findings suggest that privileged context should be treated as part of OPSD's stability design because it is associated with how far and in what direction the policy moves.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.07842 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:19:24.793Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).