Metraipolis › archive › dc_42800311cdc1
DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving — arXiv:2608.30386
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.30386
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T07:52:37.873Z
- SHA-256 of the archived text
341a1a069e5c2f9ae51eb5bdf097e6a26d391c07d514824b37caf8a256af3e57- Stable URL of this page
https://metraipolis.com/documento/dc_42800311cdc1
How to cite
DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving — arXiv:2608.30386 — arXiv (en), https://arxiv.org/abs/2608.30386 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_42800311cdc1 (Metraipolis archive dc_42800311cdc1, sha256 341a1a069e5c2f9ae51eb5bdf097e6a26d391c07d514824b37caf8a256af3e57)
BibTeX
@misc{dc42800311cdc1,
title = {DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving — arXiv:2608.30386},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.30386},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_42800311cdc1; sha256 341a1a069e5c2f9ae51eb5bdf097e6a26d391c07d514824b37caf8a256af3e57},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.30386. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Hybrid linear-attention architectures have recently scaled to large open-weight models, offering quality competitive with full attention while substantially reducing key/value (KV) cache growth. However, their in-place recurrent-state updates complicate cache management: prefix reuse requires state checkpoints alongside full-attention KV, while storing state checkpoints in full increases memory pressure, leading to more evictions and repeated prefill. By analyzing the decay structure of Gated DeltaNet (GDN) and Kimi Delta Attention (KDA), we find that different heads and channels retain prefix information over markedly different timescales, which we term \emph{retention horizons}. This variation suggests substantial compression potential in persistent state checkpoints. Building on this observation, we introduce \emph{Decay-Aware State Compression} (DASC), which derives retention horizons from model weights, selects long-horizon state units, and packs them into a ragged state checkpoint layout. To integrate efficiently with tensor-parallel inference engines, DASC furtherly balances compressed state checkpoints across TP ranks. On reuse, DASC either zero-fills omitted units or refreshes them from a bounded suffix with additional compute cost. Across retrieval and end-to-end reasoning benchmarks on Kimi-Linear, conservative DASC configurations remain close to full caching while compressing KDA recurrent state checkpoints by $2.63\times$. Under fixed state checkpoint memory budgets, the resulting capacity gains reduce mean Time to First Token (TTFT) by 42.6\% and improve input throughput by 68.4\%. At larger compression ratio, suffix refresh recovers much of the accuracy lost to more aggressive omission, at the cost of additional replay computation. Qwen with GDN exhibits a similar quality--efficiency trend, showing that DASC extends from channel-wise KDA to head-wise GDN.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.30386 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:28:26.860Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).