Metraipolis › archive › dc_9e9b06d993bd

MORTIS: Compiler-Guided KV-Cache Management for LLM Agent Programs — arXiv:2611.08638

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.08638
License (as archived)
CC0-1.0
Archived
2026-11-09T22:47:58.880Z
SHA-256 of the archived text
a8b36c496a5fe6ee1132a396cf7369c721ec54ce9ea9157311de0ef5410727a5
Stable URL of this page
https://metraipolis.com/documento/dc_9e9b06d993bd

How to cite

MORTIS: Compiler-Guided KV-Cache Management for LLM Agent Programs — arXiv:2611.08638 — arXiv (en), https://arxiv.org/abs/2611.08638 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9e9b06d993bd (Metraipolis archive dc_9e9b06d993bd, sha256 a8b36c496a5fe6ee1132a396cf7369c721ec54ce9ea9157311de0ef5410727a5)
BibTeX
@misc{dc9e9b06d993bd,
  title = {MORTIS: Compiler-Guided KV-Cache Management for LLM Agent Programs — arXiv:2611.08638},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.08638},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9e9b06d993bd; sha256 a8b36c496a5fe6ee1132a396cf7369c721ec54ce9ea9157311de0ef5410727a5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08638. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Existing agent serving systems divide each prompt into a reusable prefix and an opaque suffix, managing the KV-cache at this coarse granularity without distinguishing the components within each part. We observe that prompt components differ in which LLM calls share their values and how long those values remain reusable, a property we term intra-prompt heterogeneity. Our key insight is that, in frameworks that generate prompts from typed LLM function calls, these components correspond to program variables, allowing compiler analysis to recover their sharing patterns and lifetimes. We present MORTIS, a compiler-serving co-design that combines this analysis with runtime observations to improve KV-cache management. MORTIS reorders prompt components to expose additional prefix reuse and uses their predicted reuse and lifetimes to guide cache retention, eviction, and prefetching. Our evaluation shows that MORTIS achieves up to 1.99x job completion time (JCT) speedup and 1.97x throughput improvement over SGLang, with geometric-mean speedups of 1.45x and 1.49x, respectively.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.OS, nuovi lavori del giorno).
Categorie arXiv: Operating Systems (cs.OS).
Pagina del lavoro: https://arxiv.org/abs/2611.08638 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-09T22:41:59.952Z dalla pagina https://arxiv.org/list/cs.OS/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).