Metraipolis › archive › dc_0ff04a2aa1e1

Editing Large Language Models with Geometry-Aware Regularization — arXiv:2611.06185

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.06185
License (as archived)
CC0-1.0
Archived
2026-11-05T18:17:47.617Z
SHA-256 of the archived text
476bb28fc39a3fe30942a9d7bf9f21fe1b30a62bc81f5b8cb8f9a52d25a22ba3
Stable URL of this page
https://metraipolis.com/documento/dc_0ff04a2aa1e1

How to cite

Editing Large Language Models with Geometry-Aware Regularization — arXiv:2611.06185 — arXiv (en), https://arxiv.org/abs/2611.06185 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0ff04a2aa1e1 (Metraipolis archive dc_0ff04a2aa1e1, sha256 476bb28fc39a3fe30942a9d7bf9f21fe1b30a62bc81f5b8cb8f9a52d25a22ba3)
BibTeX
@misc{dc0ff04a2aa1e1,
  title = {Editing Large Language Models with Geometry-Aware Regularization — arXiv:2611.06185},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.06185},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0ff04a2aa1e1; sha256 476bb28fc39a3fe30942a9d7bf9f21fe1b30a62bc81f5b8cb8f9a52d25a22ba3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.06185. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Knowledge editing has become a key technique for updating knowledge in large language models. However, existing methods neglect the geometric structure of the knowledge representation space, leading to two key limitations: they cannot distinguish which pre-trained knowledge should be preferentially preserved (core vs. non-core) nor which new edits are inherently difficult (hard vs. easy). To address this, we propose GEdit, a geometry-aware framework built on a Low-Rank Plus Diagonal (LR+D) factor model that captures the intrinsic low-rank structure of Transformer FFN hidden representations. GEdit features two complementary mechanisms. First, the low-rank regularization applies targeted protection on a low-rank core knowledge subspace. Second, the adaptive regularization dynamically adjusts the regularization strength based on each edit request's editability: relaxing constraints for hard edits to ensure success, and tightening them for easy edits to better preserve pretrained knowledge. Extensive experiments on GPT2-XL, GPT-J, and LLaMA-3-8B under large-scale editing scenarios, including both mini-batch and fully sequential editing settings with up to 10,000 continuous updates, show that GEdit consistently and significantly outperforms state-of-the-art baselines in both editing success and knowledge preservation.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.06185 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-05T18:12:49.049Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).