Metraipolis › archive › dc_7995de6c3003

Evaluating the accuracy of KV cache reuse techniques — arXiv:2609.31415

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.31415
License (as archived): CC0-1.0
Archived: 2026-10-08T02:13:15.269Z
SHA-256 of the archived text: ad0ee1026d82d44b28816aeb1f5f2fa0870a43264f570e1ef9a422d3f9a380ac
Stable URL of this page: https://metraipolis.com/documento/dc_7995de6c3003

How to cite

Evaluating the accuracy of KV cache reuse techniques — arXiv:2609.31415 — arXiv (en), https://arxiv.org/abs/2609.31415 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7995de6c3003 (Metraipolis archive dc_7995de6c3003, sha256 ad0ee1026d82d44b28816aeb1f5f2fa0870a43264f570e1ef9a422d3f9a380ac)

BibTeX

@misc{dc7995de6c3003,
  title = {Evaluating the accuracy of KV cache reuse techniques — arXiv:2609.31415},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31415},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7995de6c3003; sha256 ad0ee1026d82d44b28816aeb1f5f2fa0870a43264f570e1ef9a422d3f9a380ac},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31415. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Position-independent KV cache reuse aims to reduce latency in retrieval-augmented generation by reusing chunk-level KV caches across prompts. We show that current evaluations of KV cache reuse techniques rely on measurements that fail to faithfully capture the loss of accuracy attributable to reuse, often artificially inflating the reported effectiveness. We also show that existing datasets do not exhibit the reuse dynamics needed to thoroughly evaluate such techniques. To address these issues, we propose an evaluation methodology that measures this accuracy loss without ambiguity and we introduce Boxoffice, a tool that programmatically generates evaluation datasets that exercise challenging KV cache reuse patterns.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.31415 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:43:06.752Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).