Metraipolis › archive › dc_bbadd895c831
Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.03955
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T18:53:52.663Z
- SHA-256 of the archived text
0cf9f8e451815227c8e04d6908190bcbe057ec31f811b9869c6a7f1aa7d367bd- Stable URL of this page
https://metraipolis.com/documento/dc_bbadd895c831
How to cite
Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955 — arXiv (en), https://arxiv.org/abs/2611.03955 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bbadd895c831 (Metraipolis archive dc_bbadd895c831, sha256 0cf9f8e451815227c8e04d6908190bcbe057ec31f811b9869c6a7f1aa7d367bd)
BibTeX
@misc{dcbbadd895c831,
title = {Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.03955},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bbadd895c831; sha256 0cf9f8e451815227c8e04d6908190bcbe057ec31f811b9869c6a7f1aa7d367bd},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03955. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) can memorize and reproduce fragments of their training data, raising privacy and legal concerns that motivate machine unlearning. Yet poisoned training examples can remain statistically detectable after unlearning, which raises the question of whether current unlearning objectives remove what the model has memorized at all. In this work, we introduce PINT, a novel algorithm for machine unlearning that uses the Hilbert-Schmidt Independence Criterion (HSIC) to reduce the statistical dependence between tokens to be forgotten and the internal representations used to predict them. We formally relate our HSIC-based objective to Exact Memorization through a computable upper bound, connecting the training objective directly to the model's ability to reproduce the forgotten tokens. In extensive experiments, we compare forgetting-utility trade-offs across eight settings with models up to 7B parameters and demonstrate scalability to 70B parameter models. Our evaluations cover unlearning Gaussian poisons, personal information, hazardous knowledge in biology and cybersecurity, and copyrighted books and news articles. PINT achieves the strongest forgetting-utility trade-off on average and is the only method to make Gaussian poisons undetectable while maintaining near-original model utility. Code is available at: this https URL Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.03955 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T18:35:37.408Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).