Metraipolis › archive › dc_9f445a6cda45

Collective Communication for Distributed LLM Systems: Planning, Runtime Adaptation, and Computation Coordination — arXiv:2608.15118

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.15118
License (as archived): CC0-1.0
Archived: 2026-10-18T19:02:03.722Z
SHA-256 of the archived text: 8a64651ddbb2428761f4033ff04d17f598038665ed24d45052104cdd26f76ea9
Stable URL of this page: https://metraipolis.com/documento/dc_9f445a6cda45

How to cite

Collective Communication for Distributed LLM Systems: Planning, Runtime Adaptation, and Computation Coordination — arXiv:2608.15118 — arXiv (en), https://arxiv.org/abs/2608.15118 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9f445a6cda45 (Metraipolis archive dc_9f445a6cda45, sha256 8a64651ddbb2428761f4033ff04d17f598038665ed24d45052104cdd26f76ea9)

BibTeX

@misc{dc9f445a6cda45,
  title = {Collective Communication for Distributed LLM Systems: Planning, Runtime Adaptation, and Computation Coordination — arXiv:2608.15118},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.15118},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9f445a6cda45; sha256 8a64651ddbb2428761f4033ff04d17f598038665ed24d45052104cdd26f76ea9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15118. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Distributed large language model (LLM) systems increasingly rely on collective communication primitives such as AllReduce (AR), ReduceScatter (RS), AllGather (AG), and AlltoAll (A2A). In modern LLM training and serving clusters, heterogeneous GPU interconnects, multi-NIC networking, mixed parallelism strategies, low-latency inference requests, and high-throughput training pipelines have motivated increasingly diverse ways to plan, execute, and overlap collective communication. This paper presents a tutorial-style, collective-centric taxonomy for collective communication. We organize recent advances into three layers: communication planning, which generates topology-aware collective schedules; communication execution and adaptation, which maps these schedules onto GPU runtimes and hardware in real clusters; and computation-communication coordination, which turns collective optimization into end-to-end training and inference benefits. We further discuss open challenges and future opportunities for collective communication in distributed LLM systems.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-18 in cs.DC).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2608.15118 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T18:07:16.064Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).