Metraipolis › archive › dc_b8d90199e4bf

Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.03955
License (as archived)
CC0-1.0
Archived
2026-11-04T21:50:52.980Z
SHA-256 of the archived text
6a5140e9b471eabf9ac51a6b20a0dbf6c846fd6ed46c786204a8a44d2bb71f96
Stable URL of this page
https://metraipolis.com/documento/dc_b8d90199e4bf

How to cite

Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955 — arXiv (en), https://arxiv.org/abs/2611.03955 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b8d90199e4bf (Metraipolis archive dc_b8d90199e4bf, sha256 6a5140e9b471eabf9ac51a6b20a0dbf6c846fd6ed46c786204a8a44d2bb71f96)
BibTeX
@misc{dcb8d90199e4bf,
  title = {Forget What Comes Next: LLM Unlearning via Statistical Independence — arXiv:2611.03955},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03955},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b8d90199e4bf; sha256 6a5140e9b471eabf9ac51a6b20a0dbf6c846fd6ed46c786204a8a44d2bb71f96},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03955. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) can memorize and reproduce fragments of their training data, raising privacy and legal concerns that motivate machine unlearning. Yet poisoned training examples can remain statistically detectable after unlearning, which raises the question of whether current unlearning objectives remove what the model has memorized at all. In this work, we introduce PINT, a novel algorithm for machine unlearning that uses the Hilbert-Schmidt Independence Criterion (HSIC) to reduce the statistical dependence between tokens to be forgotten and the internal representations used to predict them. We formally relate our HSIC-based objective to Exact Memorization through a computable upper bound, connecting the training objective directly to the model's ability to reproduce the forgotten tokens. In extensive experiments, we compare forgetting-utility trade-offs across eight settings with models up to 7B parameters and demonstrate scalability to 70B parameter models. Our evaluations cover unlearning Gaussian poisons, personal information, hazardous knowledge in biology and cybersecurity, and copyrighted books and news articles. PINT achieves the strongest forgetting-utility trade-off on average and is the only method to make Gaussian poisons undetectable while maintaining near-original model utility. Code is available at: this https URL

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.03955 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T21:38:49.682Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).