Metraipolis › archive › dc_4a08dc47871f
Language Models Share a Readout Bottleneck in Working Memory — arXiv:2610.21176
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.21176
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T09:14:23.802Z
- SHA-256 of the archived text
dcf82f84ad43299ab4ef9a515ea956d42bf0e71d3b22a56c5c5a544ccbb75716- Stable URL of this page
https://metraipolis.com/documento/dc_4a08dc47871f
How to cite
Language Models Share a Readout Bottleneck in Working Memory — arXiv:2610.21176 — arXiv (en), https://arxiv.org/abs/2610.21176 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4a08dc47871f (Metraipolis archive dc_4a08dc47871f, sha256 dcf82f84ad43299ab4ef9a515ea956d42bf0e71d3b22a56c5c5a544ccbb75716)
BibTeX
@misc{dc4a08dc47871f,
title = {Language Models Share a Readout Bottleneck in Working Memory — arXiv:2610.21176},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.21176},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4a08dc47871f; sha256 dcf82f84ad43299ab4ef9a515ea956d42bf0e71d3b22a56c5c5a544ccbb75716},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21176. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) reproduce a human working-memory signature on the N-back task: performance declines as memory load N increases, and errors concentrate on lures, items that repeat the current stimulus at a neighboring but incorrect serial position. We ask whether this shared behavioral signature reflects a shared mechanism. The literature offers two candidate accounts. On a precision account, softmax attention cannot concentrate its weight on the item N positions back, so adjacent items are retrieved instead. On an interference account, attention selects the correct item, but the content of competing items corrupts the decision. We dissociate these accounts in 15 pretrained LLMs from three families (70 million to 14 billion parameters) and in minimal transformers trained from scratch. In all three families, the attention heads that carry the task attend to the target on most missed matches. Linear probes decode near-perfectly whether a matching letter occurred at the target or at a lure position, even on trials where the model false-alarms to the lure, although its own decisions barely distinguish the two. Sharpening attention yields negligible gains, whereas suppressing the letter identity of non-target items improves sensitivity (d') in nearly all model-by-load conditions. In larger models the task is spread over more heads, and removing interference helps more. In minimal transformers, precision limits a single attention layer, but a second layer removes this constraint. The working-memory limit of LLMs therefore lies in reading items out under interference rather than in locating them, consistent with interference accounts of human working memory. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Neurons and Cognition (q-bio.NC). Pagina del lavoro: https://arxiv.org/abs/2610.21176 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:44:17.916Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).