Metraipolis › archive › dc_a7a003b5945a

Decoupling Knowledge and Privacy: Post-Task Self-Distillation Replay for LLM Continual Learning — arXiv:2609.29711

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.29711
License (as archived)
CC0-1.0
Archived
2026-10-08T04:27:43.248Z
SHA-256 of the archived text
a8eb7fbe8ef5a169aed6865f023490549389d1681966b669448b178b79013260
Stable URL of this page
https://metraipolis.com/documento/dc_a7a003b5945a

How to cite

Decoupling Knowledge and Privacy: Post-Task Self-Distillation Replay for LLM Continual Learning — arXiv:2609.29711 — arXiv (en), https://arxiv.org/abs/2609.29711 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a7a003b5945a (Metraipolis archive dc_a7a003b5945a, sha256 a8eb7fbe8ef5a169aed6865f023490549389d1681966b669448b178b79013260)
BibTeX
@misc{dca7a003b5945a,
  title = {Decoupling Knowledge and Privacy: Post-Task Self-Distillation Replay for LLM Continual Learning — arXiv:2609.29711},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.29711},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a7a003b5945a; sha256 a8eb7fbe8ef5a169aed6865f023490549389d1681966b669448b178b79013260},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29711. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Privacy-preserving continual learning (PPCL) must reduce the reproduction of sensitive content while retaining useful knowledge across sequential tasks. Formal privacy guarantees characterize randomized mechanisms, whereas operational output control concerns whether a trained model selectively reduces the likelihood of sensitive content in its outputs. In this work, we investigate the latter together with continual-learning utility under realistic task evolution. Retention and privacy correction operate at different granularities: task acquisition requires broad preservation of current- and old-task behavior, whereas privacy correction targets sparse annotated positions. Joint optimization leaves the current-task preservation target continually changing. We propose SPARK, a retention-correction decomposition that first freezes the learned post-task distribution and then applies selective correction around this stable reference. Self-Distillation Replay learns the current task while distilling behavior from previous tasks, and Post-Task Privacy Correction reduces annotated-PII likelihood while anchoring current- and old-task non-PII behavior to the resulting checkpoint. Extensive evaluations demonstrate that SPARK achieves effective selective PII suppression while preserving strong continual-learning utility and knowledge retention across diverse settings. Code and data will be released upon publication.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.29711 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:58.137Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).