Metraipolis › archive › dc_72baee11075e

Crossflow: Prefill-Decode Elasticity for Agentic LLM Serving — arXiv:2609.27085

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.27085
License (as archived): CC0-1.0
Archived: 2026-10-08T09:12:09.380Z
SHA-256 of the archived text: 9c7cc6ebfe3dd2b5527fd4790f5491f8d63a1d702de816a020601c452eec7dad
Stable URL of this page: https://metraipolis.com/documento/dc_72baee11075e

How to cite

Crossflow: Prefill-Decode Elasticity for Agentic LLM Serving — arXiv:2609.27085 — arXiv (en), https://arxiv.org/abs/2609.27085 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_72baee11075e (Metraipolis archive dc_72baee11075e, sha256 9c7cc6ebfe3dd2b5527fd4790f5491f8d63a1d702de816a020601c452eec7dad)

BibTeX

@misc{dc72baee11075e,
  title = {Crossflow: Prefill-Decode Elasticity for Agentic LLM Serving — arXiv:2609.27085},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.27085},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_72baee11075e; sha256 9c7cc6ebfe3dd2b5527fd4790f5491f8d63a1d702de816a020601c452eec7dad},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.27085. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As serving capacity demand surpasses that of training, serving efficiency becomes increasingly important. Prefill-decode (P/D) disaggregation improves serving efficiency through specialization and isolation of the two phases. These benefits rest on a static partitioning. Phase demand, however, is not static. We observe that in a large LLM fleet the ratio of uncached input to output tokens has peak-to-mean ratios up to 4.7x at minute timescales, and that in a public agentic trace the hourly ratio spans a median 24.5x within a single day, while reassigning a replica takes tens of minutes. Agentic traffic sharpens the mismatch. Sizing each pool at its ninety-fifth percentile leaves up to 17% of cluster capacity unused; sizing below it converts the same imbalance into queueing and unrealized throughput. We present Crossflow, which makes this boundary elastic without changing node roles. Each decode node publishes a short-lived, revocable lease that bounds local-prefill compute, KV capacity, transfer work, and projected output. Across public and internal traces, Crossflow improves token throughput by 16.2-17.4% on geometric mean over static P/D, and by up to 43.4% at high load, while reducing mean TTFT at every evaluated point.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.AI).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.27085 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T08:13:26.615Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).