Metraipolis › archive › dc_2199c159b145
Does Execution Require Target KV Fidelity? A Mixed-Fidelity KV Runtime for LLM Serving — arXiv:2609.33536
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.33536
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T22:37:33.496Z
- SHA-256 of the archived text
44ee72dd3a1c4396dc49fbebffbd10816fb3705638ff28f53471cdd184285c55- Stable URL of this page
https://metraipolis.com/documento/dc_2199c159b145
How to cite
Does Execution Require Target KV Fidelity? A Mixed-Fidelity KV Runtime for LLM Serving — arXiv:2609.33536 — arXiv (en), https://arxiv.org/abs/2609.33536 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2199c159b145 (Metraipolis archive dc_2199c159b145, sha256 44ee72dd3a1c4396dc49fbebffbd10816fb3705638ff28f53471cdd184285c55)
BibTeX
@misc{dc2199c159b145,
title = {Does Execution Require Target KV Fidelity? A Mixed-Fidelity KV Runtime for LLM Serving — arXiv:2609.33536},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.33536},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2199c159b145; sha256 44ee72dd3a1c4396dc49fbebffbd10816fb3705638ff28f53471cdd184285c55},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33536. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language model (LLM) serving is increasingly constrained by the GPU memory consumed by key-value (KV) caches. Existing compression, eviction, and offloading techniques alleviate this pressure, but serving runtimes typically treat only the configured target KV representation as execution-ready. Under memory pressure, this target-only contract can turn KV shortage into request stalls and preemptions. We present ElasticKV, a mixed-fidelity KV runtime built on the observation that target fidelity need not gate execution. ElasticKV introduces a compact intermediate KV state, making fidelity a runtime-managed execution property. To realize this state in a paged serving runtime, ElasticKV combines (i) a pair-structured layout that turns fidelity reduction into reusable GPU capacity, (ii) a dual-mode attention backend that directly consumes the compact state while preserving the native target-only path, and (iii) pressure-aware fidelity management that adapts KV fidelity to memory pressure. Our extensive evaluation across diverse workloads, model families and scales, and GPU platforms demonstrates the effectiveness and generality of ElasticKV. Under high concurrency, ElasticKV achieves 3.8-4.0$\times$ lower time-to-first-token (TTFT) and 9.1$\times$ lower P90 TTFT than vLLM while preserving generation quality. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.33536 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T22:14:27.978Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).