Metraipolis › archive › dc_069eb0ed787e
AnchorKV: Anchor-Residual KV Cache Compression — arXiv:2608.02901
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.02901
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T12:31:19.046Z
- SHA-256 of the archived text
5dd9e20a6df60fb02997f494582e5fb6de3002a3d4f90cb2abb70a02a3e9a266- Stable URL of this page
https://metraipolis.com/documento/dc_069eb0ed787e
How to cite
AnchorKV: Anchor-Residual KV Cache Compression — arXiv:2608.02901 — arXiv (en), https://arxiv.org/abs/2608.02901 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_069eb0ed787e (Metraipolis archive dc_069eb0ed787e, sha256 5dd9e20a6df60fb02997f494582e5fb6de3002a3d4f90cb2abb70a02a3e9a266)
BibTeX
@misc{dc069eb0ed787e,
title = {AnchorKV: Anchor-Residual KV Cache Compression — arXiv:2608.02901},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.02901},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_069eb0ed787e; sha256 5dd9e20a6df60fb02997f494582e5fb6de3002a3d4f90cb2abb70a02a3e9a266},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.02901. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The key-value (KV) cache is the primary memory bottleneck in long-context LLM inference. Existing approaches attack it from opposite ends: eviction methods permanently discard tokens, degrading performance whenever a discarded token later proves essential, while quantization methods retain all tokens at low precision but offer limited compression. We propose AnchorKV, a compression scheme that shrinks the cache by $20\times$ without discarding a single token. AnchorKV represents the cache using a small set of anchors stored exactly, expresses every other token through its most similar anchor, and refines only those whose approximation most affects the model's output. AnchorKV consistently preserves accuracy across models and datasets, retaining 99% of the full-cache score at the 70B scale, while keeping the entire context at a fraction of its cost. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.CL). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.02901 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T12:17:37.797Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).