Metraipolis › archive › dc_d187cdeada8f

Joint Effects of GPU Server Topology, Parallelism, and Congestion Control on MoE Inference: A Controlled Simulation Study — arXiv:2609.37828

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.37828
License (as archived): CC0-1.0
Archived: 2026-10-07T16:59:10.865Z
SHA-256 of the archived text: 807a073ca91cdc42690d17c64ce191a093c46b4767aecfbf4b8e4a79676138db
Stable URL of this page: https://metraipolis.com/documento/dc_d187cdeada8f

How to cite

Joint Effects of GPU Server Topology, Parallelism, and Congestion Control on MoE Inference: A Controlled Simulation Study — arXiv:2609.37828 — arXiv (en), https://arxiv.org/abs/2609.37828 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d187cdeada8f (Metraipolis archive dc_d187cdeada8f, sha256 807a073ca91cdc42690d17c64ce191a093c46b4767aecfbf4b8e4a79676138db)

BibTeX

@misc{dcd187cdeada8f,
  title = {Joint Effects of GPU Server Topology, Parallelism, and Congestion Control on MoE Inference: A Controlled Simulation Study — arXiv:2609.37828},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37828},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d187cdeada8f; sha256 807a073ca91cdc42690d17c64ce191a093c46b4767aecfbf4b8e4a79676138db},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37828. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Mixture-of-experts (MoE) models expand capacity via sparse activation, but inference across GPUs introduces tensor-parallel (TP) collectives and expert-parallel (EP) dispatch and combine operations. Completion time depends not just on communication volume but on how logical groups map onto intra-server interconnects, GPU--NIC connections, and the inter-node network. Using ASTRA-sim with the NS-3 discrete-event backend, we build a controlled matrix of 32 GPU ranks with data and pipeline parallelism fixed at one. Workloads are fixed-length 4096-token prefill-like synthetic Chakra traces from four MoE configurations. Experiments cover six server topologies, four TP/EP partitions, two TP collective algorithms, and four network/congestion-control modes, yielding 768 deterministic simulations. In the 144-configuration feedback-enabled subset per model, exposed communication accounts for 89.9%--95.8% of mean completion time. TP16EP2 requires 3.68--4.35x the mean completion time of TP2EP16. With fixed rank mapping, ASTRA-sim Double Binary Tree (DBT) incurs 28.3%--83.2% more time than Ring. InfiniBand-like High Precision Congestion Control (HPCC) is ~0.9% lower than HPCC over RDMA over Converged Ethernet (RoCE), whereas RoCE with Data Center Quantized Congestion Notification (DCQCN) is 23.8%--35.7% slower than RoCE HPCC. Topology effects are conditional: Topology~6 leads at low TP degrees but loses its advantage at high TP degrees, and additional GPUs or NICs help only when rank mapping balances traffic across injection paths. Under uniform 32-way sharding, the largest checkpoint-weight shard is ~48.75 GB per rank, so all configurations meet a 64 GB per-accelerator weight-residency criterion. Within the evaluated workload and simulator semantics, server topology, parallelism, collective implementation, and congestion control jointly determine exposed communication and completion time.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-30 in cs.DC).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2609.37828 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T16:31:32.463Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).