Metraipolis › archive › dc_008b42aee3dd
QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding — arXiv:2608.05326
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.05326
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-25T21:30:51.152Z
- SHA-256 of the archived text
907657d31fe489fbbe6830107c9cadb9246a63f0d6a81a96f5482125cd5b5269- Stable URL of this page
https://metraipolis.com/documento/dc_008b42aee3dd
How to cite
QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding — arXiv:2608.05326 — arXiv (en), https://arxiv.org/abs/2608.05326 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_008b42aee3dd (Metraipolis archive dc_008b42aee3dd, sha256 907657d31fe489fbbe6830107c9cadb9246a63f0d6a81a96f5482125cd5b5269)
BibTeX
@misc{dc008b42aee3dd,
title = {QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding — arXiv:2608.05326},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.05326},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_008b42aee3dd; sha256 907657d31fe489fbbe6830107c9cadb9246a63f0d6a81a96f5482125cd5b5269},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.05326. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Autoregressive large language model inference is increasingly constrained by the memory footprint of the Key-Value (KV) cache. A dominant line of work reduces this footprint by evicting tokens that appear unimportant under attention-derived scores. However, such policies make an implicit irreversible decision: once a token is evicted, it cannot become useful again. We show that this assumption is brittle during decoding. Token and window importance drift as generated queries evolve, causing standard eviction policies to permanently discard states that later receive substantial attention under the full-cache model. To characterize this behaviour, we introduce Future Missed Mass and Global LIR, two diagnostics that measure future attention assigned to discarded states and the reactivation of historically inactive regions. We propose QEvict, a three-tier KV-cache management scheme that replaces binary retain-or-delete eviction with recoverable eviction. QEvict maintains high-confidence windows in full precision, stores intermediate windows in a quantized recoverable tier, and deletes only the lowest-confidence windows. During decoding, cumulative attention scores update window importance and when a quantized window becomes important again, it is dequantized and promoted to the full-precision. Under a fixed memory budget, this design preserves broader historical context while retaining exact full precision for the most important regions. Across long-context understanding, retrieval, and reasoning benchmarks, QEvict consistently improves over representative eviction and quantization baselines, reducing missed attention and improving information retention Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-07 in cs.CL). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.05326 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-25T21:24:38.244Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).