Metraipolis › archive › dc_bbadd895c831

Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.03955
License (as archived)
CC0-1.0
Archived
2026-11-04T18:53:52.663Z
SHA-256 of the archived text
0cf9f8e451815227c8e04d6908190bcbe057ec31f811b9869c6a7f1aa7d367bd
Stable URL of this page
https://metraipolis.com/documento/dc_bbadd895c831

How to cite

Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955 — arXiv (en), https://arxiv.org/abs/2611.03955 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bbadd895c831 (Metraipolis archive dc_bbadd895c831, sha256 0cf9f8e451815227c8e04d6908190bcbe057ec31f811b9869c6a7f1aa7d367bd)
BibTeX
@misc{dcbbadd895c831,
  title = {Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03955},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bbadd895c831; sha256 0cf9f8e451815227c8e04d6908190bcbe057ec31f811b9869c6a7f1aa7d367bd},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03955. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) can memorize and reproduce fragments of their training data, raising privacy and legal concerns that motivate machine unlearning. Yet poisoned training examples can remain statistically detectable after unlearning, which raises the question of whether current unlearning objectives remove what the model has memorized at all. In this work, we introduce PINT, a novel algorithm for machine unlearning that uses the Hilbert-Schmidt Independence Criterion (HSIC) to reduce the statistical dependence between tokens to be forgotten and the internal representations used to predict them. We formally relate our HSIC-based objective to Exact Memorization through a computable upper bound, connecting the training objective directly to the model's ability to reproduce the forgotten tokens. In extensive experiments, we compare forgetting-utility trade-offs across eight settings with models up to 7B parameters and demonstrate scalability to 70B parameter models. Our evaluations cover unlearning Gaussian poisons, personal information, hazardous knowledge in biology and cybersecurity, and copyrighted books and news articles. PINT achieves the strongest forgetting-utility trade-off on average and is the only method to make Gaussian poisons undetectable while maintaining near-original model utility. Code is available at: this https URL

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.03955 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T18:35:37.408Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).