Metraipolis › archive › dc_50dcdc9ed1c2
Adaptive KV Retention for LLM Agents at Human-Approval Timescales — arXiv:2608.30830
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.30830
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T10:13:59.935Z
- SHA-256 of the archived text
3cf92febdcd78d69b6b0bc36553f616f16e63807054d355fb5dcc0ecbfaa1dc3- Stable URL of this page
https://metraipolis.com/documento/dc_50dcdc9ed1c2
How to cite
Adaptive KV Retention for LLM Agents at Human-Approval Timescales — arXiv:2608.30830 — arXiv (en), https://arxiv.org/abs/2608.30830 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_50dcdc9ed1c2 (Metraipolis archive dc_50dcdc9ed1c2, sha256 3cf92febdcd78d69b6b0bc36553f616f16e63807054d355fb5dcc0ecbfaa1dc3)
BibTeX
@misc{dc50dcdc9ed1c2,
title = {Adaptive KV Retention for LLM Agents at Human-Approval Timescales — arXiv:2608.30830},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.30830},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_50dcdc9ed1c2; sha256 3cf92febdcd78d69b6b0bc36553f616f16e63807054d355fb5dcc0ecbfaa1dc3},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.30830. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Unlike the seconds-scale tool-call pauses targeted by prior agent-serving systems, agentic LLM requests can be suspended for minutes or hours while waiting for human approval. We study how suspension and resumption affect GPU serving performance and develop a retention policy that balances active-serving capacity against future recomputation under uncertain approval waits. The central tension is severe because retaining suspended KV preserves fast resume but can consume enough GPU capacity to reduce active-serving goodput by 41%, while evicting it avoids that residency cost at the expense of nearly $10\times$ higher resume latency when the request returns. We develop a tiered retention controller around GPU opportunity cost, which expresses the serving capacity consumed by preserving or reconstructing a suspended request's KV state in a common GPU-time cost. Within host memory, the controller selects between indefinite retention and load-indexed expiration using calibration wait samples, without requiring per-request wait prediction. On human-scale approval workloads, our controller improves active-request goodput by 23-51% over the vLLM baselines, 22-29% over MORI, and 41-52% over Continuum. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-01 in cs.OS). Categorie arXiv: Operating Systems (cs.OS). Pagina del lavoro: https://arxiv.org/abs/2608.30830 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T10:03:18.184Z dalla pagina https://arxiv.org/catchup/cs.OS/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).