Metraipolis › archive › dc_b9ee6b2829ac
TRAP: Understanding and Mitigating Privacy Memorization in Language Models — arXiv:2609.32293
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.32293
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T21:00:02.566Z
- SHA-256 of the archived text
d08b383ad4c862d4941ac11800fe1b7933fab5f2761f5f7872c37db9e81e137d- Stable URL of this page
https://metraipolis.com/documento/dc_b9ee6b2829ac
How to cite
TRAP: Understanding and Mitigating Privacy Memorization in Language Models — arXiv:2609.32293 — arXiv (en), https://arxiv.org/abs/2609.32293 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b9ee6b2829ac (Metraipolis archive dc_b9ee6b2829ac, sha256 d08b383ad4c862d4941ac11800fe1b7933fab5f2761f5f7872c37db9e81e137d)
BibTeX
@misc{dcb9ee6b2829ac,
title = {TRAP: Understanding and Mitigating Privacy Memorization in Language Models — arXiv:2609.32293},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.32293},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b9ee6b2829ac; sha256 d08b383ad4c862d4941ac11800fe1b7933fab5f2761f5f7872c37db9e81e137d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32293. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Fine-tuning a language model on sensitive records can leave it able to reproduce them. We ask when this memorization arises and how to prevent it without knowing in advance which spans are sensitive. Our starting point is that most memorization scores and attacks share one statistical core: whether the model assigns a token more probability than some reference would. Taking as the reference a model trained on the complementary half of the same corpus gives the Target Reference Advantage (TRA), a per-token signal that separates what a model fit to a particular record from what it learned across records, and is cheap and differentiable. We then study what drives memorization during fine-tuning: it keeps growing well past the validation minimum, is larger on small datasets and at higher learning rates, and higher when the underlying task is harder. Early stopping removes much of it, but because it is chosen by aggregate validation loss it helps least for rare, hard-to-predict spans embedded in otherwise learnable text, which is exactly what sensitive information tends to be. We therefore introduce TRAP, a one-sided penalty on tokenwise TRA that acts only where the target model pulls ahead of its reference. On student essays with annotated personal information and clinical cases with patient identifiers, TRAP brings memorization near the level of an untrained model at little utility cost, where generic regularizers barely move and differential privacy gives up most of what fine-tuning bought. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.32293 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:02.283Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).