Metraipolis › archive › dc_bcbef4b16822
Inside the Trap: A Renewal Law for Prefix-Cache Collapse in LLM Agent Serving — arXiv:2611.08902
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.08902
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-09T22:36:29.862Z
- SHA-256 of the archived text
ad7b186c7bc5cf6d29fe163eb7199632770154fdd4f52443aa8dfa307be9dd33- Stable URL of this page
https://metraipolis.com/documento/dc_bcbef4b16822
How to cite
Inside the Trap: A Renewal Law for Prefix-Cache Collapse in LLM Agent Serving — arXiv:2611.08902 — arXiv (en), https://arxiv.org/abs/2611.08902 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bcbef4b16822 (Metraipolis archive dc_bcbef4b16822, sha256 ad7b186c7bc5cf6d29fe163eb7199632770154fdd4f52443aa8dfa307be9dd33)
BibTeX
@misc{dcbcbef4b16822,
title = {Inside the Trap: A Renewal Law for Prefix-Cache Collapse in LLM Agent Serving — arXiv:2611.08902},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.08902},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bcbef4b16822; sha256 ad7b186c7bc5cf6d29fe163eb7199632770154fdd4f52443aa8dfa307be9dd33},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08902. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Prefix caching lets an LLM agent's follow-up turns reuse cached KV, but a burst can tip an engine into a trap in which every turn misses and the backlog does not drain, at a load it served before. In registered paired tests on two engines, two GPUs and two models (passed at 9 of 10 rates), a 200-session burst left the engine at zero hits to the end in all 30 runs, while every matched no-burst run stayed good and a session cap prevented the trap in all 30 capped runs. We analyse an idealization of the trapped state exactly: for a fluid FIFO server on which every turn misses, we prove that along FIFO orbits its cycles obey a linear renewal recursion. As is classical, the trap drains if and only if arrivals are below the all-miss capacity; above it, a Perron root fixes the dynamics: every trapped state converges to a known self-similar regime in which the gap between turns grows linearly and the turns served are young and short. For our workload the backlog grows at 0.6 to 0.8 of the excess load and sessions complete faster than the all-miss capacity, a youth premium with turn-count and cost-growth factors, so in the model a trap's measured throughput overstates the recovery threshold. Retrospectively, with nothing fitted to trap data, the theory matches 57 stuck runs: the mean served prompt within 3% in 50 runs, completions above the all-miss capacity in every run, and the pooled gap growth. Blind in outcome, 4 runs trapped by a tool slowdown met 4 of 5 pre-set scores, cut traps persisted, and with a 20-step budget, trapped runs completed below the all-miss capacity, as predicted. Of two exits, refusing arrivals sheds sessions; a session cap restores reuse: in registered tests it rescued 9 of 9 established traps at the same load without shedding, and the engine stayed good after release. We do not prove that the engine converges to the model. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Performance (cs.PF). Pagina del lavoro: https://arxiv.org/abs/2611.08902 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-09T22:31:17.040Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).