Metraipolis › archive › dc_86c030c03acf

Tessera: Demand-Driven KV Cache Management for Retrieval-Augmented LLM Serving — arXiv:2609.32999

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32999
License (as archived)
CC0-1.0
Archived
2026-10-07T22:12:01.257Z
SHA-256 of the archived text
724fbae139d2ba58ffb998645a4cacf20a3be2f4266723b767eb0eb815b8afbb
Stable URL of this page
https://metraipolis.com/documento/dc_86c030c03acf

How to cite

Tessera: Demand-Driven KV Cache Management for Retrieval-Augmented LLM Serving — arXiv:2609.32999 — arXiv (en), https://arxiv.org/abs/2609.32999 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_86c030c03acf (Metraipolis archive dc_86c030c03acf, sha256 724fbae139d2ba58ffb998645a4cacf20a3be2f4266723b767eb0eb815b8afbb)
BibTeX
@misc{dc86c030c03acf,
  title = {Tessera: Demand-Driven KV Cache Management for Retrieval-Augmented LLM Serving — arXiv:2609.32999},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32999},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_86c030c03acf; sha256 724fbae139d2ba58ffb998645a4cacf20a3be2f4266723b767eb0eb815b8afbb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32999. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

RAG and retrieval-based agent memory both inject retrieved content into LLM prompts, as document chunks and recalled memory records, respectively. The same content can recur across requests at different prompt positions or after different preceding contexts, preventing reuse through conventional prefix caching. Our characterization finds that records recurring outside the matching prefix account for over 70% of injected memory tokens in agent-memory workloads. Composable KV-reuse methods enable reuse in such cases, but online serving introduces a management problem: a recurring unit's KV states may not yet exist, may have been evicted, or may reside on another node. We present Tessera, a disaggregated serving system that makes retrieval the control plane for KV reuse. By exposing the context units needed before model execution, retrieval allows Tessera to combine current demand with retrieval history, KV residency, and generation load to coordinate cache management and request routing. Generation nodes concurrently prepare locally cached, remotely cached, and missing states, while retaining newly computed states off the request's critical path. Across RAG and agent-memory workloads, Tessera lowers mean TTFT by up to 3.6x over SGLang and LMCache with EPIC at matched request rates, and sustains low TTFT at rates where the baselines saturate, while matching the answer quality of the underlying composition policy.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-29 in cs.DC).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2609.32999 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T21:53:13.658Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).