Metraipolis › archive › dc_752c8eafc3e2

Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA — arXiv:2506.20856

Source
arXiv (en)
Original
https://arxiv.org/abs/2506.20856
License (as archived)
CC0-1.0
Archived
2026-11-03T08:02:16.016Z
SHA-256 of the archived text
9ab8d86b59291d76083df660d0651e3f9a01bc7bd3400d03c269d0813e7ac2ee
Stable URL of this page
https://metraipolis.com/documento/dc_752c8eafc3e2

How to cite

Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA — arXiv:2506.20856 — arXiv (en), https://arxiv.org/abs/2506.20856 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_752c8eafc3e2 (Metraipolis archive dc_752c8eafc3e2, sha256 9ab8d86b59291d76083df660d0651e3f9a01bc7bd3400d03c269d0813e7ac2ee)
BibTeX
@misc{dc752c8eafc3e2,
  title = {Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA — arXiv:2506.20856},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2506.20856},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_752c8eafc3e2; sha256 9ab8d86b59291d76083df660d0651e3f9a01bc7bd3400d03c269d0813e7ac2ee},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2506.20856. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Memorization in large language models (LLMs) makes them vulnerable to data extraction attacks. While pre-training memorization has been extensively studied, fewer works have explored its impact in fine-tuning, particularly for LoRA fine-tuning, a widely adopted parameter-efficient method. In this work, we re-examine memorization in fine-tuning and uncover a surprising divergence from prior findings across different fine-tuning strategies. Factors such as model scale and data duplication, which strongly influence memorization in pre-training and full fine-tuning, do not follow the same trend in LoRA fine-tuning. Using a more relaxed similarity-based memorization metric, we demonstrate that LoRA significantly reduces memorization risks compared to full fine-tuning, while still maintaining strong task performance.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2506.20856 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T07:59:38.757Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).