Metraipolis › archive › dc_0c4e6ec67cce
DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization — arXiv:2608.27513
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.27513
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T10:47:47.578Z
- SHA-256 of the archived text
a41df030a32036365255c41c45f775a5de6ff576866fb9016654ce739331abca- Stable URL of this page
https://metraipolis.com/documento/dc_0c4e6ec67cce
How to cite
DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization — arXiv:2608.27513 — arXiv (en), https://arxiv.org/abs/2608.27513 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0c4e6ec67cce (Metraipolis archive dc_0c4e6ec67cce, sha256 a41df030a32036365255c41c45f775a5de6ff576866fb9016654ce739331abca)
BibTeX
@misc{dc0c4e6ec67cce,
title = {DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization — arXiv:2608.27513},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.27513},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0c4e6ec67cce; sha256 a41df030a32036365255c41c45f775a5de6ff576866fb9016654ce739331abca},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.27513. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Complex reasoning and agentic applications increasingly rely on long-context inference, where growing KV caches increase both memory usage and decoding overhead. Hybrid models reduce these costs by combining Softmax Attention with Gated DeltaNet (GDN) or Kimi Delta Attention (KDA), which maintain fixed-size recurrent states. These states are commonly stored in FP32 and consume substantial GPU memory, while their updates are limited by memory bandwidth. Quantization can reduce both storage footprint and memory traffic, but we find that uniform INT8 and FP8 degrade complex reasoning accuracy, while INT4 and NVFP4 collapse it to near zero. To our knowledge, this is the first study of post-training recurrent-state quantization for GDN and KDA. Our analysis reveals that outliers in GDN and KDA states are concentrated in particular key channels and value dimensions. Learned decay influences how much quantization error is retained. We find that largely the same GDN heads and KDA key channels exhibit slow decay across tasks. Based on these insights, we propose DAMP, which jointly considers quantization error and decay-based error retention to select high-risk key channels offline. Under a fixed storage budget, it retains these channels in FP16 and stores the remainder in INT8. We evaluate DAMP on Qwen3.6-35B, Kimi-Linear-48B and Kimi-K3 across six reasoning and code generation benchmarks. At 9.9 bits per state value, DAMP maintains average accuracy close to FP32. In SGLang, DAMP reduces recurrent-state storage by 69.1%, accelerates the recurrent-state update kernel by up to 2.59x , and lowers full-model time per output token by up to 19.0%. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-31 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.27513 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T10:33:42.449Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-31?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).