Metraipolis › archive › dc_406d52a9be61
Budgeted Cache Repair for Cross-Context KV-Cache Reuse — arXiv:2610.02233
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.02233
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-05T22:51:52.108Z
- SHA-256 of the archived text
37d5f23c68197cbe7cad46ca816b3fb3aa5ee3df552ab62a1d9a3a1c5e249596- Stable URL of this page
https://metraipolis.com/documento/dc_406d52a9be61
How to cite
Budgeted Cache Repair for Cross-Context KV-Cache Reuse — arXiv:2610.02233 — arXiv, https://arxiv.org/abs/2610.02233 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_406d52a9be61 (Metraipolis archive dc_406d52a9be61, sha256 37d5f23c68197cbe7cad46ca816b3fb3aa5ee3df552ab62a1d9a3a1c5e249596)
BibTeX
@misc{dc406d52a9be61,
title = {Budgeted Cache Repair for Cross-Context KV-Cache Reuse — arXiv:2610.02233},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.02233},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_406d52a9be61; sha256 37d5f23c68197cbe7cad46ca816b3fb3aa5ee3df552ab62a1d9a3a1c5e249596},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02233. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Cross-context KV-cache reuse predicts a shared segment's keys and values under a new prefix instead of recomputing them, and has been reported to do so without quality loss. We find otherwise, and identify two problems. (1) A hidden cost: on MMLU and GSM8K, reuse costs substantial accuracy. (2) A decision at the wrong unit: no rule for deciding whether to reuse a cache removes that cost. What does help is choosing which parts of the cache to recompute, and the value of choosing well falls as the unit of choice grows: informed selection removes 49.5% of the cache error beyond chance at single rows (one token's keys and values), 10.6% at 64-token chunks, and nothing at the level of whole calls. Budgeted Cache Repair (BCR) acts at the unit where selection still pays. It drafts two tokens from the assembled cache, ranks cache rows by the attention those tokens pay them, and recomputes a fixed budget of rows exactly, in one of three layouts. The cost is paid rather than predicted away, and the draft that fails as a gate succeeds as a selector. BCR restores GSM8K to dense-prefill accuracy while still serving most calls from cache, and its best layout outperforms every reuse baseline's mean in the reference grid. The draft also beats a coin-flip selector at the same budget - a control prior evaluations lack. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Hardware Architecture (cs.AR) ; Computation and Language (cs.CL); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.02233 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-05T22:46:10.599Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).