Metraipolis › archive › dc_e0541edaa66e
D-Quant: Driftable Entropy Coding for KV Cache Quantization — arXiv:2609.19880
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.19880
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T03:48:07.836Z
- SHA-256 of the archived text
3ca5bca301a99288281d04d5ece52b304460a2116442172ec3aaa5631bb1761a- Stable URL of this page
https://metraipolis.com/documento/dc_e0541edaa66e
How to cite
D-Quant: Driftable Entropy Coding for KV Cache Quantization — arXiv:2609.19880 — arXiv (en), https://arxiv.org/abs/2609.19880 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e0541edaa66e (Metraipolis archive dc_e0541edaa66e, sha256 3ca5bca301a99288281d04d5ece52b304460a2116442172ec3aaa5631bb1761a)
BibTeX
@misc{dce0541edaa66e,
title = {D-Quant: Driftable Entropy Coding for KV Cache Quantization — arXiv:2609.19880},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.19880},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e0541edaa66e; sha256 3ca5bca301a99288281d04d5ece52b304460a2116442172ec3aaa5631bb1761a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19880. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The KV cache has become a major bottleneck in deploying LLMs, as its memory footprint grows linearly with sequence length and batch size, imposing substantial pressure on both memory capacity and bandwidth. Among various KV cache compression techniques, quantization is particularly attractive due to its effectiveness and ease of deployment. However, most existing methods rely on fixed-width quantization, where a $b$ bit representation is inherently limited to $2^b$ quantization levels. As the bit width decreases, the number of available levels shrinks exponentially, leading to severe information loss and rapid performance degradation. We further observe that fixed-width quantization fails to exploit the highly non-uniform distribution of KV cache. After rotation and normalization, KV values approximately follow a normal distribution, with most values concentrated near the center and only a small fraction appearing in the tails. Nevertheless, fixed-width coding allocates the same number of bits to frequent and rare symbols. Entropy coding naturally exploits such non-uniformity by assigning shorter codewords to frequent symbols and longer ones to rare symbols, substantially reducing the average number of bits required for representation. However, its variable-length output is not suited to highly parallel attention kernels, where efficient dequantization and computation rely on regular memory layouts and fixed-stride accesses. To bridge this gap, we propose \textbf{D-Quant}, a flexible KV cache quantization framework that introduces a \textbf{drift} mechanism to convert entropy-coded representations of each token into fixed-size bitstreams, enabling regular memory access and parallel dequantization within attention kernels.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.19880 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T03:24:08.139Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).