Metraipolis › archive › dc_dd450bb74435

Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.03955
License (as archived)
CC0-1.0
Archived
2026-11-04T12:39:24.694Z
SHA-256 of the archived text
025eaa4964d8e82740bace53aee6801b78861e890888c7186aa23f94b5740725
Stable URL of this page
https://metraipolis.com/documento/dc_dd450bb74435

How to cite

Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955 — arXiv (en), https://arxiv.org/abs/2611.03955 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dd450bb74435 (Metraipolis archive dc_dd450bb74435, sha256 025eaa4964d8e82740bace53aee6801b78861e890888c7186aa23f94b5740725)
BibTeX
@misc{dcdd450bb74435,
  title = {Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03955},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dd450bb74435; sha256 025eaa4964d8e82740bace53aee6801b78861e890888c7186aa23f94b5740725},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03955. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) can memorize and reproduce fragments of their training data, raising privacy and legal concerns that motivate machine unlearning. Yet poisoned training examples can remain statistically detectable after unlearning, which raises the question of whether current unlearning objectives remove what the model has memorized at all. In this work, we introduce PINT, a novel algorithm for machine unlearning that uses the Hilbert-Schmidt Independence Criterion (HSIC) to reduce the statistical dependence between tokens to be forgotten and the internal representations used to predict them. We formally relate our HSIC-based objective to Exact Memorization through a computable upper bound, connecting the training objective directly to the model's ability to reproduce the forgotten tokens. In extensive experiments, we compare forgetting-utility trade-offs across eight settings with models up to 7B parameters and demonstrate scalability to 70B parameter models. Our evaluations cover unlearning Gaussian poisons, personal information, hazardous knowledge in biology and cybersecurity, and copyrighted books and news articles. PINT achieves the strongest forgetting-utility trade-off on average and is the only method to make Gaussian poisons undetectable while maintaining near-original model utility. Code is available at: this https URL

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.03955 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T12:29:47.258Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).