Metraipolis › archive › dc_ebdff7561171

EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory — arXiv:2610.10533

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.10533
License (as archived)
CC0-1.0
Archived
2026-10-08T11:26:25.812Z
SHA-256 of the archived text
9d81e4fa24cedecc470d1bfa2ebc195091c6a5d82be2cecfef240faa5489a9d7
Stable URL of this page
https://metraipolis.com/documento/dc_ebdff7561171

How to cite

EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory — arXiv:2610.10533 — arXiv (en), https://arxiv.org/abs/2610.10533 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ebdff7561171 (Metraipolis archive dc_ebdff7561171, sha256 9d81e4fa24cedecc470d1bfa2ebc195091c6a5d82be2cecfef240faa5489a9d7)
BibTeX
@misc{dcebdff7561171,
  title = {EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory — arXiv:2610.10533},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.10533},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ebdff7561171; sha256 9d81e4fa24cedecc470d1bfa2ebc195091c6a5d82be2cecfef240faa5489a9d7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.10533. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Beyond model scaling, this architecture has demonstrated the potential to decouple factual knowledge storage from general-purpose computation, offering a promising route to updating factual knowledge while keeping the Transformer backbone fixed. Realizing this potential is challenging because different expressions of a fact may activate different n-gram embeddings, while updating shared embeddings can unintentionally change the model's predictions about other facts. We propose EngramEdit for decoupled knowledge updates through conditional memory. EngramEdit first computes target memory representations that make the model predict the updated fact across multiple expressions. It then jointly updates the shared n-gram embeddings to match these targets across expressions and edits, penalizing updates to frequently reused embeddings more strongly to preserve unrelated knowledge. Experiments show that EngramEdit enables independent factual knowledge updates through conditional memory, achieving near-perfect editing success. Revised knowledge is usable across unseen expressions and in multi-hop reasoning, with nearly three times the strongest baseline's accuracy under chain-of-thought (CoT) prompting. Unrelated knowledge and general capabilities are largely preserved even as factual updates accumulate. These findings show that EngramEdit turns conditional memory into an editable knowledge interface, extending its role beyond model scaling to support decoupled knowledge updates.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.10533 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T10:24:06.357Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).