Metraipolis › archive › dc_a152e51351bc

Distillation Defenses Easily Break After Reinforcement Learning — arXiv:2609.35699

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.35699
License (as archived)
CC0-1.0
Archived
2026-10-07T21:03:53.713Z
SHA-256 of the archived text
0167636a1c6a9053783194502349662b61848a6ed9ce6bd4c234b10014fcb700
Stable URL of this page
https://metraipolis.com/documento/dc_a152e51351bc

How to cite

Distillation Defenses Easily Break After Reinforcement Learning — arXiv:2609.35699 — arXiv (en), https://arxiv.org/abs/2609.35699 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a152e51351bc (Metraipolis archive dc_a152e51351bc, sha256 0167636a1c6a9053783194502349662b61848a6ed9ce6bd4c234b10014fcb700)
BibTeX
@misc{dca152e51351bc,
  title = {Distillation Defenses Easily Break After Reinforcement Learning — arXiv:2609.35699},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35699},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a152e51351bc; sha256 0167636a1c6a9053783194502349662b61848a6ed9ce6bd4c234b10014fcb700},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35699. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Distillation attacks copy the reasoning capabilities of closed-source large language models, allowing bad actors to replicate state-of-the-art performance at low cost. Attackers systematically collect a large volume of frontier model reasoning traces and then train (i.e., "distill") their own models on these traces. Existing defenses against distillation attacks are typically evaluated immediately after distillation, implicitly assuming attackers do not train their models any further. In this paper, we argue that a more realistic threat model includes further training with reinforcement learning after distillation. A misspecified threat model can give a false sense of security -- some defenses that seem effective after distillation can be broken after subsequent reinforcement learning. Practically, reinforcement learning lowers the bar for a distillation attack to be effective. We show that simple attacks can steal reasoning capabilities from existing closed-source language models using data easily obtainable from current APIs, yielding reasoning improvements equivalent to more sophisticated attacks that extract the full hidden traces. Results indicate that any distillation defense that leaks sufficient information to reconstruct approximate reasoning traces is likely ineffective. We conclude by discussing broader implications and batch-level distillation defenses which could be more effective.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2609.35699 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:03.430Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).