Metraipolis › archive › dc_50731048ec00

Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA — arXiv:2506.20856

Source: arXiv (en)
Original: https://arxiv.org/abs/2506.20856
License (as archived): CC0-1.0
Archived: 2026-11-03T07:32:26.106Z
SHA-256 of the archived text: d3dbcb983d5334e5477bef3075dcb5135643940a2198c442658a9917466e1ab3
Stable URL of this page: https://metraipolis.com/documento/dc_50731048ec00

How to cite

Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA — arXiv:2506.20856 — arXiv (en), https://arxiv.org/abs/2506.20856 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_50731048ec00 (Metraipolis archive dc_50731048ec00, sha256 d3dbcb983d5334e5477bef3075dcb5135643940a2198c442658a9917466e1ab3)

BibTeX

@misc{dc50731048ec00,
  title = {Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA — arXiv:2506.20856},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2506.20856},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_50731048ec00; sha256 d3dbcb983d5334e5477bef3075dcb5135643940a2198c442658a9917466e1ab3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2506.20856. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Memorization in large language models (LLMs) makes them vulnerable to data extraction attacks. While pre-training memorization has been extensively studied, fewer works have explored its impact in fine-tuning, particularly for LoRA fine-tuning, a widely adopted parameter-efficient method. In this work, we re-examine memorization in fine-tuning and uncover a surprising divergence from prior findings across different fine-tuning strategies. Factors such as model scale and data duplication, which strongly influence memorization in pre-training and full fine-tuning, do not follow the same trend in LoRA fine-tuning. Using a more relaxed similarity-based memorization metric, we demonstrate that LoRA significantly reduces memorization risks compared to full fine-tuning, while still maintaining strong task performance.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2506.20856 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T07:29:33.668Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).