Metraipolis › archive › dc_406d52a9be61

Budgeted Cache Repair for Cross-Context KV-Cache Reuse — arXiv:2610.02233

Source
arXiv
Original
https://arxiv.org/abs/2610.02233
License (as archived)
CC0-1.0
Archived
2026-10-05T22:51:52.108Z
SHA-256 of the archived text
37d5f23c68197cbe7cad46ca816b3fb3aa5ee3df552ab62a1d9a3a1c5e249596
Stable URL of this page
https://metraipolis.com/documento/dc_406d52a9be61

How to cite

Budgeted Cache Repair for Cross-Context KV-Cache Reuse — arXiv:2610.02233 — arXiv, https://arxiv.org/abs/2610.02233 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_406d52a9be61 (Metraipolis archive dc_406d52a9be61, sha256 37d5f23c68197cbe7cad46ca816b3fb3aa5ee3df552ab62a1d9a3a1c5e249596)
BibTeX
@misc{dc406d52a9be61,
  title = {Budgeted Cache Repair for Cross-Context KV-Cache Reuse — arXiv:2610.02233},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.02233},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_406d52a9be61; sha256 37d5f23c68197cbe7cad46ca816b3fb3aa5ee3df552ab62a1d9a3a1c5e249596},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02233. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Cross-context KV-cache reuse predicts a shared segment's keys and values under a new prefix instead of recomputing them, and has been reported to do so without quality loss. We find otherwise, and identify two problems. (1) A hidden cost: on MMLU and GSM8K, reuse costs substantial accuracy. (2) A decision at the wrong unit: no rule for deciding whether to reuse a cache removes that cost. What does help is choosing which parts of the cache to recompute, and the value of choosing well falls as the unit of choice grows: informed selection removes 49.5% of the cache error beyond chance at single rows (one token's keys and values), 10.6% at 64-token chunks, and nothing at the level of whole calls. Budgeted Cache Repair (BCR) acts at the unit where selection still pays. It drafts two tokens from the assembled cache, ranks cache rows by the attention those tokens pay them, and recomputes a fixed budget of rows exactly, in one of three layouts. The cost is paid rather than predicted away, and the draft that fails as a gate succeeds as a selector. BCR restores GSM8K to dense-prefill accuracy while still serving most calls from cache, and its best layout outperforms every reuse baseline's mean in the reference grid. The draft also beats a coin-flip selector at the same budget - a control prior evaluations lack.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Hardware Architecture (cs.AR) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.02233 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T22:46:10.599Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).