Metraipolis › archive › dc_2ac820c42ffd

Where KV-Cache Error Lives in Looped Language Models: A Diagnostic Study of Ouro and Huginn — arXiv:2610.41825

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.41825
License (as archived)
CC0-1.0
Archived
2026-11-02T07:44:56.177Z
SHA-256 of the archived text
e20f871cb676f85f19391e8b71dd03d525aa5a0def467989360980b2cce01118
Stable URL of this page
https://metraipolis.com/documento/dc_2ac820c42ffd

How to cite

Where KV-Cache Error Lives in Looped Language Models: A Diagnostic Study of Ouro and Huginn — arXiv:2610.41825 — arXiv (en), https://arxiv.org/abs/2610.41825 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2ac820c42ffd (Metraipolis archive dc_2ac820c42ffd, sha256 e20f871cb676f85f19391e8b71dd03d525aa5a0def467989360980b2cce01118)
BibTeX
@misc{dc2ac820c42ffd,
  title = {Where KV-Cache Error Lives in Looped Language Models: A Diagnostic Study of Ouro and Huginn — arXiv:2610.41825},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41825},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2ac820c42ffd; sha256 e20f871cb676f85f19391e8b71dd03d525aa5a0def467989360980b2cce01118},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41825. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A looped language model applies one block of layers 4 times per token and keeps a key/value (KV) plane for each loop. We study where storage error changes the next-token distribution and how shared-weight error accumulates, using Ouro-1.4B (4 loops) on GSM8K and MATH and Huginn-0125 (32 loops) on GSM8K-derived chains, with dense, simulated quantization. Per unit of error, values carry 94% of the diagonal Fisher mass, but measured KL is dominated by keys with larger quantization errors. Local second-order prediction follows the measured KL of typical sequences, including 136 held-out chains, but misses a few confident rows, and a local second-order factor for shared weights at 32 loops ranges from 1.65 (INT8) to 4.09 (INT4), so it depends on the error direction. A cell-level allocator lowers one-step KL at 25-48% of BF16 bytes, while paired task-accuracy comparisons depend on budget and the 25% gap against K4V2 shrinks to near zero under a post hoc answer-extraction check. A later Ouro-2.6B test retains the key/value split but falsifies measured-cost allocation's advantage over Fisher at 31%; across all 21 ledger registrations, 60 of 176 criteria failed.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2610.41825 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T07:37:30.165Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).