Metraipolis › archive › dc_eea5d3552219

Monte Carlo Estimation for KV Cache Eviction — arXiv:2610.07643

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.07643
License (as archived)
CC0-1.0
Archived
2026-10-07T11:20:57.414Z
SHA-256 of the archived text
74ede2771567884ea734bfdeff729065731f4d1d2bc2cc41e494274e3cfac61a
Stable URL of this page
https://metraipolis.com/documento/dc_eea5d3552219

How to cite

Monte Carlo Estimation for KV Cache Eviction — arXiv:2610.07643 — arXiv (en), https://arxiv.org/abs/2610.07643 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_eea5d3552219 (Metraipolis archive dc_eea5d3552219, sha256 74ede2771567884ea734bfdeff729065731f4d1d2bc2cc41e494274e3cfac61a)
BibTeX
@misc{dceea5d3552219,
  title = {Monte Carlo Estimation for KV Cache Eviction — arXiv:2610.07643},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.07643},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_eea5d3552219; sha256 74ede2771567884ea734bfdeff729065731f4d1d2bc2cc41e494274e3cfac61a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.07643. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Most KV-cache eviction methods ask, in effect, which memory appeared important while reading the prompt? We instead ask, which memory will matter while answering? Since decoding queries are unavailable at eviction time, prior future-aware methods rely on pseudo-responses or synthetic future-query estimates. We cast fixed-budget future-aware eviction as distributional estimation over plausible model-conditional query trajectories and introduce LORE-KV (Lookahead Output-perturbation with Reliability-weighted Ensembles for Key-Value caches), a training-free method that samples short autoregressive continuations from the frozen target model and uses their response-side query states to estimate prompt-token utility. Tokens are scored by projected leave-one-out attention-output deletion cost and aggregated across sampled futures with optional trajectory weighting. The temporary continuations are discarded before final decoding, requiring no auxiliary model or training. Ablations isolate the mechanism: at B=128, a single response-side continuation recovers about 89% of the gain over the prompt-window control, while additional futures provide smaller improvements. At B=128, LORE-KV raises the LongBench average on Qwen2.5-14B from 45.49 to 48.24 (+2.75) and the 16K RULER average on Mistral-7B from 45.20 to 51.05 (+5.85). Gains diminish at larger cache budgets and coexist with task-level regressions. LORE-KV incurs 1.46-2.77x AnDPro's per-sample wall-clock time as a one-time compression overhead across six dense and hybrid-attention backbones.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.07643 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:15:24.743Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).