Metraipolis › archive › dc_6abcc2b72e81

Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling Agents — arXiv:2610.05833

Source
arXiv
Original
https://arxiv.org/abs/2610.05833
License (as archived)
CC0-1.0
Archived
2026-10-06T05:25:22.724Z
SHA-256 of the archived text
92179d3d700fdf1cae8f57d021a51ad4a85d68001cfab00f8be9e87b1071af17
Stable URL of this page
https://metraipolis.com/documento/dc_6abcc2b72e81

How to cite

Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling Agents — arXiv:2610.05833 — arXiv, https://arxiv.org/abs/2610.05833 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6abcc2b72e81 (Metraipolis archive dc_6abcc2b72e81, sha256 92179d3d700fdf1cae8f57d021a51ad4a85d68001cfab00f8be9e87b1071af17)
BibTeX
@misc{dc6abcc2b72e81,
  title = {Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling Agents — arXiv:2610.05833},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.05833},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6abcc2b72e81; sha256 92179d3d700fdf1cae8f57d021a51ad4a85d68001cfab00f8be9e87b1071af17},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05833. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long-running agents repeatedly call an LLM while retaining most of their document window, evicting old documents, and appending new ones. These rolling updates break exact prefix caching and motivate non-prefix KV-cache reuse with selective recomputation. We show that persistent KV-cache reuse with selective recomputation can be history-dependent: in our rolling-agent workload, an unchanged prompt can produce different answers depending on the requests processed before it. At a matched 5% recomputation budget, document-aligned recomputation reduces answer variation across request orders from 69.0% with CacheBlend's token top-$k$ policy to 26.1%. When each prompt is evaluated after a different sequence of preceding requests, document-aligned recomputation improves fidelity to full prefill by 34.5-52.5 percentage points over token top-$k$, while both policies achieve approximately 5.7$\times$ median TTFT speedup. Our ablation study shows that, in our rolling-agent workload, contiguity is the main factor associated with robust selective recomputation.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2610.05833 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:05.638Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).