Metraipolis › archive › dc_18ec69909acd
PDD: Unleashing Economical and Flexible Heterogeneous LLM Inference via Cross-Datacenter Prefill-Decode Disaggregation — arXiv:2609.13161
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.13161
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T23:17:43.770Z
- SHA-256 of the archived text
c9ebf751e0aa037c634e49d60b1e0abe90547a9d29c239161d95ea388db9df33- Stable URL of this page
https://metraipolis.com/documento/dc_18ec69909acd
How to cite
PDD: Unleashing Economical and Flexible Heterogeneous LLM Inference via Cross-Datacenter Prefill-Decode Disaggregation — arXiv:2609.13161 — arXiv (en), https://arxiv.org/abs/2609.13161 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_18ec69909acd (Metraipolis archive dc_18ec69909acd, sha256 c9ebf751e0aa037c634e49d60b1e0abe90547a9d29c239161d95ea388db9df33)
BibTeX
@misc{dc18ec69909acd,
title = {PDD: Unleashing Economical and Flexible Heterogeneous LLM Inference via Cross-Datacenter Prefill-Decode Disaggregation — arXiv:2609.13161},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.13161},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_18ec69909acd; sha256 c9ebf751e0aa037c634e49d60b1e0abe90547a9d29c239161d95ea388db9df33},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.13161. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Interconnecting geographically dispersed clusters over wide-area Ethernet provides a scalable and cost-effective alternative to dedicated intra-datacenter heterogeneous clusters for Large Language Model (LLM) inference. However, this cross-datacenter disaggregation imposes heavy KV-cache transfers between clusters, causing substantial Time-to-First-Token (TTFT) latency, which is especially detrimental for agentic workloads characterized by long contexts, high cache hit rates, and short outputs. We propose PDD, a three-tier disaggregation architecture built upon prefill-decode (PD) disaggregation, consisting of Prefill, RelayDecode (RLD), and MainDecode (MD) instances. On Cluster A, Prefill instances perform the prefill computation, while RLD instances immediately receive the KV cache via high-speed RDMA and begin decoding, thereby overlapping the KV transfer to Cluster B over TCP-based Ethernet. MD instances on Cluster B receive the KV cache along with the tokens produced by RLD, and decoding is then seamlessly handed off from RLD to MD for completion. To maximize overall efficiency, PDD employs three core mechanisms: Decode-side RadixCache to alleviate bandwidth bottlenecks, an Extend-Decode Handoff mechanism for smooth control migration between RLD and MD, and multi-stage pipeline orchestration to manage complex inter-tier dependencies under high concurrency and long-term serving. We further design a low-cost, fine-grained heterogeneous deployment scheme that maximizes latency-masking efficiency at marginal cost. Compared to the intra-DC homogeneous PD baseline, PDD's cross-datacenter mapping of compute-intensive H100s and memory-bandwidth-optimized H200s achieves a Benefit-Cost Ratio (BCR) up to 37.5% higher in SLA-compliant goodput. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-15 in cs.DC). Categorie arXiv: Hardware Architecture (cs.AR) ; Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2609.13161 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:54:13.001Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).