Metraipolis › archive › dc_0db8edf76771
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization — arXiv:2609.38169
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.38169
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:30:10.012Z
- SHA-256 of the archived text
713d362d3b94788194d75176c8700029fad4be34d74d868224c6c8ab808508fb- Stable URL of this page
https://metraipolis.com/documento/dc_0db8edf76771
How to cite
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization — arXiv:2609.38169 — arXiv (en), https://arxiv.org/abs/2609.38169 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0db8edf76771 (Metraipolis archive dc_0db8edf76771, sha256 713d362d3b94788194d75176c8700029fad4be34d74d868224c6c8ab808508fb)
BibTeX
@misc{dc0db8edf76771,
title = {STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization — arXiv:2609.38169},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.38169},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0db8edf76771; sha256 713d362d3b94788194d75176c8700029fad4be34d74d868224c6c8ab808508fb},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38169. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Linear attention replaces growing KV caches with fixed-size recurrent states, yet these persistent states can become a substantial memory bottleneck under concurrent serving. Directly quantizing recurrent states to low precision often leads to severe accuracy degradation, as quantization errors propagate through successive state updates. We discover that the impact of these errors depends on two complementary dimensions: temporally, errors in long-lived memory can persist across many decoding steps; spatially, errors in different key rows affect model outputs differently, while state magnitudes vary substantially along both rows and columns. Motivated by these observations, we propose STEPQuant, a spatial-temporal post-training quantization framework for Delta-rule recurrent states. STEPQuant allocates precision according to error magnitude and memory lifetime, and jointly fits key-row and value-column scales based on state distributions and key-row impact on output error. Experiments on Qwen3.8-27B and Kimi-Linear-48B-A3B-Instruct across both long- and short-generation benchmarks show that STEPQuant closely matches FP32-state accuracy under a nominal 6-bit budget and outperforms uniform INT8 in its 4-bit configuration. Integrated into SGLang with optimized GPU kernels, 6-bit STEPQuant achieves over 5x recurrent-state compression and reduces total serving memory by up to 68.7%. Our code is available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.38169 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T15:54:54.076Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).