Metraipolis › archive › dc_ee6642eacfbb

Lachesis: Lifetime-Aware KV Cache Placement for Agent Serving across HBM and High-Bandwidth Flash — arXiv:2610.08378

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.08378
License (as archived): CC0-1.0
Archived: 2026-10-07T11:24:50.352Z
SHA-256 of the archived text: 956f2e8ea47559f70bf339658b56567d39f5999a0000de7bc93b950f5286587b
Stable URL of this page: https://metraipolis.com/documento/dc_ee6642eacfbb

How to cite

Lachesis: Lifetime-Aware KV Cache Placement for Agent Serving across HBM and High-Bandwidth Flash — arXiv:2610.08378 — arXiv (en), https://arxiv.org/abs/2610.08378 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ee6642eacfbb (Metraipolis archive dc_ee6642eacfbb, sha256 956f2e8ea47559f70bf339658b56567d39f5999a0000de7bc93b950f5286587b)

BibTeX

@misc{dcee6642eacfbb,
  title = {Lachesis: Lifetime-Aware KV Cache Placement for Agent Serving across HBM and High-Bandwidth Flash — arXiv:2610.08378},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.08378},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ee6642eacfbb; sha256 956f2e8ea47559f70bf339658b56567d39f5999a0000de7bc93b950f5286587b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08378. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model (LLM) serving is increasingly dominated by agentic workloads, in which agents and their sub-agents accumulate context as KV cache across many requests, consuming substantial memory. High-bandwidth flash (HBF) is a promising solution, providing an order of magnitude greater capacity at HBM-class read bandwidth, but its finite write endurance is the key limiting factor. Our key insight is that KV cache should be placed across HBM and HBF by its lifetime. Placing shorter-lived data in HBM lets HBM absorb more of an agent run's writes and sends less of them to HBF. As the lifetime of KV cache in agentic serving is dictated by the harness, the program that orchestrates the agents, we analyze its behavior and identify three axes along which lifetime diverges, temporal, structural, and inter-worker. Guided by these observations, we present Lachesis, a lifetime-aware KV cache placement layer between the agent harness and the serving engine. At write time, it places each segment in HBM or HBF according to its lifetime, and frees its blocks once the segment is no longer read. In trace-driven simulation, Lachesis extends HBF lifetime by 1.19-3.13x over HBM-first placement, reaching 3.3-12.2 device-years. Even under continuous 24x7 operation at the full load a tight SLO admits, HBF outlasts its five-year warranty on the multi-agent trace.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Hardware Architecture (cs.AR) ; Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2610.08378 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:15:54.492Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).