Metraipolis › archive › dc_bdc2aee9f776
Kathleen Remembers: Length-Invariant One-Shot Recall Without Attention — arXiv:2608.30376
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.30376
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T08:07:30.356Z
- SHA-256 of the archived text
4a6dd2b487c5793272ff2606fbbd9a46ffdd187e40748575a57f5634f7e5d4bc- Stable URL of this page
https://metraipolis.com/documento/dc_bdc2aee9f776
How to cite
Kathleen Remembers: Length-Invariant One-Shot Recall Without Attention — arXiv:2608.30376 — arXiv (en), https://arxiv.org/abs/2608.30376 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bdc2aee9f776 (Metraipolis archive dc_bdc2aee9f776, sha256 4a6dd2b487c5793272ff2606fbbd9a46ffdd187e40748575a57f5634f7e5d4bc)
BibTeX
@misc{dcbdc2aee9f776,
title = {Kathleen Remembers: Length-Invariant One-Shot Recall Without Attention — arXiv:2608.30376},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.30376},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bdc2aee9f776; sha256 4a6dd2b487c5793272ff2606fbbd9a46ffdd187e40748575a57f5634f7e5d4bc},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.30376. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Recurrent, attention-free sequence models share a structural weakness: a fading state cannot perform exact recall of something seen once, far in the past. We add to the Kathleen trunk a second memory layer -- a "notebook": a fixed-key holographic (HRR) associative store with a learned local write gate, a self-gating raw read, and write-triggered forgetting -- 25K parameters that attach to the logits of any trunk. (1) Mechanism: on a controlled needle-in-haystack task the notebook reaches 80-82% one-shot recall at 4x the training length, where the bare trunk scores ~4% and a parameter-matched attention head scores 100% inside its training length and 0% beyond it. Addressing is length-invariant by construction; the untrained memory alone recalls at 90% accuracy identically at 512, 2048 and 4096 bytes. Because the store is a linear superposition, two capabilities follow from arithmetic alone: selective unlearning (one subtraction erases one fact to chance, retained facts unharmed) and per-token attribution (counterfactual erasure names the source fact of every correct byte, 100% provenance). (2) Real text: on WikiText-2 bytes the notebook improves prediction of repeated rare words by +0.15-0.27 bits/byte, the gain growing with the distance between mentions and holding zero-shot at 4x training length; write-triggered forgetting eliminates memory pollution at 8x length (first-mention cost +0.33 -> -0.004). (3) Scope and scale: a parameter-matched attention head does generalize on natural-text repetition, so the notebook's claim is exact recall at O(L); on a WikiText-103 ladder (8 to 512 MB) the zero-shot repeat gain rises monotonically. All experiments are pre-registered, seeds reported, and reproducible on a single free-tier GPU. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.30376 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T07:47:52.805Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).