Metraipolis › archive › dc_ca3fb6a9f392

Characterizing Parallelism Strategies in LLM Inference: Fundamental Compute-Communication Trade-offs — arXiv:2610.05305

Source
arXiv
Original
https://arxiv.org/abs/2610.05305
License (as archived)
CC0-1.0
Archived
2026-10-06T05:26:47.922Z
SHA-256 of the archived text
1078221566c9cad3f1925fc168650f5203580666d17c9a91562b03f2db3fec79
Stable URL of this page
https://metraipolis.com/documento/dc_ca3fb6a9f392

How to cite

Characterizing Parallelism Strategies in LLM Inference: Fundamental Compute-Communication Trade-offs — arXiv:2610.05305 — arXiv, https://arxiv.org/abs/2610.05305 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ca3fb6a9f392 (Metraipolis archive dc_ca3fb6a9f392, sha256 1078221566c9cad3f1925fc168650f5203580666d17c9a91562b03f2db3fec79)
BibTeX
@misc{dcca3fb6a9f392,
  title = {Characterizing Parallelism Strategies in LLM Inference: Fundamental Compute-Communication Trade-offs — arXiv:2610.05305},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.05305},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ca3fb6a9f392; sha256 1078221566c9cad3f1925fc168650f5203580666d17c9a91562b03f2db3fec79},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05305. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large Language Model (LLM) inference has become the dominant workload in modern AI systems, requiring serving infrastructures to maximize throughput while meeting strict latency Service-Level Objectives (SLOs). Since state-of-the-art LLMs exceed the compute and memory capacity of a single GPU, inference is commonly distributed across multiple GPUs using tensor parallelism (TP), pipeline parallelism (PP), or hybrid parallelism (HB). However, selecting the most effective parallelism strategy remains challenging due to complex interactions among computation, communication, pipeline utilization, sequence length, batch size, and model architecture. Existing approaches largely rely on empirical evaluation and provide limited analytical insight into the trade-offs among these strategies, particularly across the distinct prefill and decoding phases of inference. In this paper, we present a unified analytical framework for modeling distributed LLM inference under TP, PP, and HB. The framework decomposes end-to-end latency into computation, inter-GPU communication, and pipeline bubble overhead, and derives analytical models that capture TP collective communication, PP point-to-point communication, and pipeline utilization as functions of hardware, model, and workload characteristics. The model further characterizes the differing execution behavior of prefill and decoding, explaining why PP-oriented configurations favor compute-intensive prefill while TP-oriented configurations reduce decoding latency by eliminating pipeline bubbles. Experiments with modern LLMs on multi-GPU platforms validate the model and confirm the fundamental compute-communication trade-off across parallelism strategies. The framework provides practical guidance for parallelism selection, capacity planning, and optimization of future LLM serving systems.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2610.05305 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:05.993Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).