Metraipolis › archive › dc_ef642a27dad2
Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets — arXiv:2609.27981
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.27981
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T10:05:09.786Z
- SHA-256 of the archived text
ce2372674cc17949ab9304b5e3def642eb9f572328219e86946b43033028b5de- Stable URL of this page
https://metraipolis.com/documento/dc_ef642a27dad2
How to cite
Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets — arXiv:2609.27981 — arXiv (en), https://arxiv.org/abs/2609.27981 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ef642a27dad2 (Metraipolis archive dc_ef642a27dad2, sha256 ce2372674cc17949ab9304b5e3def642eb9f572328219e86946b43033028b5de)
BibTeX
@misc{dcef642a27dad2,
title = {Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets — arXiv:2609.27981},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.27981},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ef642a27dad2; sha256 ce2372674cc17949ab9304b5e3def642eb9f572328219e86946b43033028b5de},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.27981. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
KV-cache eviction is typically evaluated through average quality-memory trade-offs, yet a small average loss can hide requests whose utility degrades materially. We reformulate eviction as a deployment risk-control problem: a material degradation occurs when eviction lowers task utility by more than a deployment-specified tolerance relative to full-KV inference on the same request, and deployment risk is the population frequency of such events. Given a reliability contract specifying a target risk level and confidence requirement, we use a compressor-agnostic post-hoc certification procedure to select a retention policy from calibration data with a finite-sample guarantee, falling back to full KV when no compressed policy is certified. Across multiple eviction methods, Llama and Mistral models, and LongBench and RULER-32K, the same contract supports substantially different levels of eviction: on Llama, it certifies SnapKV at 75% retention on LongBench but no tested compressed policy on RULER-32K, triggering full-KV fallback. Policies with empirical degradation rates below the 5% target can still fail finite-sample certification; on Llama LongBench, empirical thresholding selects uncertified policies that retain 5-10 percentage points less cache across fixed-budget methods. The proposed framework converts a deployment-level reliability requirement into a KV-memory operating point. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.27981 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T09:11:49.644Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).