Metraipolis › archive › dc_62a4e0353396

Dual-QK: Sharp Queries and Flat Keys for Prunable 2-bit KV Caches — arXiv:2610.09827

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.09827
License (as archived)
CC0-1.0
Archived
2026-10-08T12:35:52.339Z
SHA-256 of the archived text
f006e6432c87ffe17772698a64ea27d5bc68a20e0c11a1969f4f5af001a77916
Stable URL of this page
https://metraipolis.com/documento/dc_62a4e0353396

How to cite

Dual-QK: Sharp Queries and Flat Keys for Prunable 2-bit KV Caches — arXiv:2610.09827 — arXiv (en), https://arxiv.org/abs/2610.09827 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_62a4e0353396 (Metraipolis archive dc_62a4e0353396, sha256 f006e6432c87ffe17772698a64ea27d5bc68a20e0c11a1969f4f5af001a77916)
BibTeX
@misc{dc62a4e0353396,
  title = {Dual-QK: Sharp Queries and Flat Keys for Prunable 2-bit KV Caches — arXiv:2610.09827},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.09827},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_62a4e0353396; sha256 f006e6432c87ffe17772698a64ea27d5bc68a20e0c11a1969f4f5af001a77916},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.09827. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long inputs and extended generation increase the storage and access costs of the key-value (KV) cache. Low-bit quantization reduces storage and memory traffic, while query-channel pruning can further reduce key-cache reads. Rotation-based quantization redistributes the energy of key outliers across channels. To maintain computational invariance, the same orthogonal transform must be applied to queries, preserving query-key dot products. However, this rotation can disperse query energy, weakening the separation between a few large components to retain and many small ones to prune. We introduce Dual-QK, which uses paired non-orthogonal query and key transforms to address this conflict. Using calibrated query and key statistics, Dual-QK combines partial key whitening with a query-aligned basis to balance key scales for INT2 quantization and concentrate query energy for dynamic channel pruning. Channel-0 protection and bucket-relative RoPE support low-bit accuracy over long contexts. Experiments on four models across five generative benchmarks and long-context retrieval tasks show improved accuracy over OSCAR on most tasks at 40% query-channel sparsity. At a 128K context, Dual-QK provides $6.8\times$ KV-cache compression and an estimated $8.3\times$ reduction in KV read volume relative to unpruned BF16. Under the evaluated configurations, our SGLang implementation achieves up to $3.75\times$ the decoding throughput of unpruned BF16.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.09827 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T12:14:15.279Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).