Metraipolis › archive › dc_3acd3ddce17e
Low-Discrepancy Dither for Quantized Recurrent State Caches — arXiv:2609.39185
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.39185
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T13:45:54.832Z
- SHA-256 of the archived text
2da48804965b0fd54bd1a2ccb1d6c63c1dc37c6e1bb451c2ba5b7663f3f1f79f- Stable URL of this page
https://metraipolis.com/documento/dc_3acd3ddce17e
How to cite
Low-Discrepancy Dither for Quantized Recurrent State Caches — arXiv:2609.39185 — arXiv (en), https://arxiv.org/abs/2609.39185 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3acd3ddce17e (Metraipolis archive dc_3acd3ddce17e, sha256 2da48804965b0fd54bd1a2ccb1d6c63c1dc37c6e1bb451c2ba5b7663f3f1f79f)
BibTeX
@misc{dc3acd3ddce17e,
title = {Low-Discrepancy Dither for Quantized Recurrent State Caches — arXiv:2609.39185},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.39185},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3acd3ddce17e; sha256 2da48804965b0fd54bd1a2ccb1d6c63c1dc37c6e1bb451c2ba5b7663f3f1f79f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39185. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Mamba-style and hybrid language models compress their past into a fixed-size recurrent state that is rewritten at every generated token. Storing this state in low precision saves memory bandwidth, but every rounding error is fed back into the next update and can accumulate over long generations. Production systems round the state stochastically; we ask which rounding rule such caches should use. We find that a deterministic golden-ratio Weyl dither, which needs no random numbers, consistently brings the quantized model closer to the full-precision one than stochastic rounding, across pure and hybrid models, storage formats, and long decoding horizons, at no extra cost. Round-to-nearest behaves differently: because it discards small updates, its error keeps growing, so it can look best in short evaluations yet falls far behind over long generations. A discrepancy analysis explains this ordering, and we document implementation pitfalls that silently remove the benefit. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.39185 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T13:18:10.332Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).