Metraipolis › archive › dc_2bfee537245d

GPU-Initiated Communication: Dissecting Down to the Bone — arXiv:2610.01380

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.01380
License (as archived): CC0-1.0
Archived: 2026-10-06T14:02:44.255Z
SHA-256 of the archived text: 5c8f4f71f4381eb71f5d5a6d08c33fdb73f160090ebf577e9306600b47b8f440
Stable URL of this page: https://metraipolis.com/documento/dc_2bfee537245d

How to cite

GPU-Initiated Communication: Dissecting Down to the Bone — arXiv:2610.01380 — arXiv (en), https://arxiv.org/abs/2610.01380 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2bfee537245d (Metraipolis archive dc_2bfee537245d, sha256 5c8f4f71f4381eb71f5d5a6d08c33fdb73f160090ebf577e9306600b47b8f440)

BibTeX

@misc{dc2bfee537245d,
  title = {GPU-Initiated Communication: Dissecting Down to the Bone — arXiv:2610.01380},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.01380},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2bfee537245d; sha256 5c8f4f71f4381eb71f5d5a6d08c33fdb73f160090ebf577e9306600b47b8f440},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01380. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

GPU-initiated communication lets GPU threads post RDMA operations directly to the NIC. It underpins NVSHMEM, NCCL GIN, and DeepEP, which serve the fine-grained, latency-critical communication of Mixture-of-Experts (MoE) models, yet its performance characteristics and optimizations remain scarcely documented beyond source code, and library comparisons fail to separate the costs of the hardware mechanism from those of the library around it. This paper dissects GPU-initiated communication at the GPU-NIC boundary. We first detail the GPU-side network path: queue placement, work-request construction, doorbell ordering, and completion semantics. We then introduce mini-gda and mini-proxy, minimal transports for the GPU and CPU-proxy submission paths, and measure them alongside NVSHMEM IBGDA, NCCL GIN, DeepEP, UCCL-EP, MSCCL++, and fabric-lib on NVIDIA H100, H200, B200, and GB200 platforms. A minimal GPU path issues an operation in 0.7 $\mu$s and completes in 4.0 $\mu$s; libraries add up to 4.6 $\mu$s of issue time through queue management, memory ordering, and completion scope, and issue time scales with the SM clock. A tuned CPU proxy matches or beats the GPU path at idle, at the cost of a dedicated core whose operating state sets its latency and throughput. On either path, sharing a queue with bulk traffic raises latency by one to three orders of magnitude. Reaching the 260 M msg/s ceiling of our InfiniBand platform requires doorbell batching and queue parallelism, and both have resource costs: communication code can reduce GPU block residency even when unused, and all-to-all traffic loses 59% of its NIC message rate at about 3,000 active connections. The submission path alone therefore does not predict communication performance. Our experiment code and results are available at this https URL .

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-10-02 in cs.DC).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Networking and Internet Architecture (cs.NI); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2610.01380 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T12:22:22.555Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).