Metraipolis › archive › dc_a72da4a44902

KV$^2$: A Self-Refining KV Cache — arXiv:2610.03198

Source
arXiv
Original
https://arxiv.org/abs/2610.03198
License (as archived)
CC0-1.0
Archived
2026-10-05T22:49:32.625Z
SHA-256 of the archived text
c7069859b48fcc794fc146a279c8dce5a3447d8efdc3c58b47b76da4978c5923
Stable URL of this page
https://metraipolis.com/documento/dc_a72da4a44902

How to cite

KV$^2$: A Self-Refining KV Cache — arXiv:2610.03198 — arXiv, https://arxiv.org/abs/2610.03198 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a72da4a44902 (Metraipolis archive dc_a72da4a44902, sha256 c7069859b48fcc794fc146a279c8dce5a3447d8efdc3c58b47b76da4978c5923)
BibTeX
@misc{dca72da4a44902,
  title = {KV$^2$: A Self-Refining KV Cache — arXiv:2610.03198},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.03198},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a72da4a44902; sha256 c7069859b48fcc794fc146a279c8dce5a3447d8efdc3c58b47b76da4978c5923},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.03198. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The memory footprint of the key-value (KV) cache constrains the practical use of long-context models, and it dominates cost when one prefilled context must later serve many different queries. In this reusable setting, query-agnostic compression trades cost against quality: lightweight estimators are cheap but less accurate, whereas full-context reconstruction scoring is more accurate yet reprocesses the entire prompt. We introduce KV$^2$, a query-agnostic KV-cache compression method based on selective reconstruction. KV$^2$ first uses a lightweight proxy scorer to identify informative in-context tokens, then reprocesses only this subset to compute final eviction scores. On RULER, Needle-in-a-Haystack, and LongBench, KV$^2$'s margin over baselines widens as the budget tightens: on RULER 16K at a 2% KV-cache budget it improves the average score over the next-best baseline by more than 40 percentage points, and on LongBench it attains the highest average across 2%-10% budgets at lower compression-stage runtime and peak memory than full-context reconstruction. Reusable KV-cache compression thus does not require reprocessing the full context. Our code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.03198 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T22:45:55.681Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).