Metraipolis › archive › dc_0ce48bf65cc4
Quantizing Recurrent Language Models Is Easier Than You Think — arXiv:2611.01864
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.01864
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T18:18:14.929Z
- SHA-256 of the archived text
adf6d57afaf5468c9ea90400e9bf3e2157c7c0a6facff9397f84c14a8aafd238- Stable URL of this page
https://metraipolis.com/documento/dc_0ce48bf65cc4
How to cite
Quantizing Recurrent Language Models Is Easier Than You Think — arXiv:2611.01864 — arXiv (en), https://arxiv.org/abs/2611.01864 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0ce48bf65cc4 (Metraipolis archive dc_0ce48bf65cc4, sha256 adf6d57afaf5468c9ea90400e9bf3e2157c7c0a6facff9397f84c14a8aafd238)
BibTeX
@misc{dc0ce48bf65cc4,
title = {Quantizing Recurrent Language Models Is Easier Than You Think — arXiv:2611.01864},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.01864},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0ce48bf65cc4; sha256 adf6d57afaf5468c9ea90400e9bf3e2157c7c0a6facff9397f84c14a8aafd238},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01864. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Linear recurrences are becoming an increasingly common building block in large language models, replacing the growing KV cache of softmax attention with a fixed-size recurrent state that enables efficient decoding. However, the quantization stack has largely been developed around softmax Transformers, whose numerical properties and inference state differ from those of recurrent models. Our goal is to understand these differences and study the design space of quantization methods for recurrent models. In our analysis, we differentiate between two targets. First, extra-state quantization concerns weights and activations outside the recurrent state, whose distributions are shaped indirectly by the sequence mixer during training. In controlled 1.3B parameter models, we find that decayed recurrences develop weaker residual outlier channels than softmax attention and are therefore easier to quantize without incoherence processing. Second, intra-state quantization concerns the recurrent state itself during decode, where repeated updates create a different numerical problem from KV cache quantization. We instantiate our findings as LoReS, a state quantization pipeline combining value-axis scale grouping to match output channel magnitude, decay-based mixed-precision to protect slowly decaying heads or channels, and stochastic rounding to limit error accumulation under repeated requantization. Across Qwen-3.5 9B, Nemotron-H 8B, and Kimi-Linear 48B-A3B, LoReS supports robust 8-bit state quantization and lower effective precision on shorter decode horizons. We implement LoReS in vLLM and demonstrate end-to-end decode speedups. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.01864 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T18:09:53.633Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).