Metraipolis › archive › dc_069eb0ed787e

AnchorKV: Anchor-Residual KV Cache Compression — arXiv:2608.02901

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.02901
License (as archived)
CC0-1.0
Archived
2026-10-31T12:31:19.046Z
SHA-256 of the archived text
5dd9e20a6df60fb02997f494582e5fb6de3002a3d4f90cb2abb70a02a3e9a266
Stable URL of this page
https://metraipolis.com/documento/dc_069eb0ed787e

How to cite

AnchorKV: Anchor-Residual KV Cache Compression — arXiv:2608.02901 — arXiv (en), https://arxiv.org/abs/2608.02901 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_069eb0ed787e (Metraipolis archive dc_069eb0ed787e, sha256 5dd9e20a6df60fb02997f494582e5fb6de3002a3d4f90cb2abb70a02a3e9a266)
BibTeX
@misc{dc069eb0ed787e,
  title = {AnchorKV: Anchor-Residual KV Cache Compression — arXiv:2608.02901},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.02901},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_069eb0ed787e; sha256 5dd9e20a6df60fb02997f494582e5fb6de3002a3d4f90cb2abb70a02a3e9a266},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.02901. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The key-value (KV) cache is the primary memory bottleneck in long-context LLM inference. Existing approaches attack it from opposite ends: eviction methods permanently discard tokens, degrading performance whenever a discarded token later proves essential, while quantization methods retain all tokens at low precision but offer limited compression. We propose AnchorKV, a compression scheme that shrinks the cache by $20\times$ without discarding a single token. AnchorKV represents the cache using a small set of anchors stored exactly, expresses every other token through its most similar anchor, and refines only those whose approximation most affects the model's output. AnchorKV consistently preserves accuracy across models and datasets, retaining 99% of the full-cache score at the 70B scale, while keeping the entire context at a fraction of its cost.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.02901 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T12:17:37.797Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).