Metraipolis › archive › dc_a1cdff6e1204
mKernel: Fast Multi-GPU, Multi-Node Fused Kernels — arXiv:2609.13585
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.13585
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T22:48:38.019Z
- SHA-256 of the archived text
8fc9acef8d16761b44b447364ef682d26a46c466c73e4b6070bb6d9f9a9b0ceb- Stable URL of this page
https://metraipolis.com/documento/dc_a1cdff6e1204
How to cite
mKernel: Fast Multi-GPU, Multi-Node Fused Kernels — arXiv:2609.13585 — arXiv (en), https://arxiv.org/abs/2609.13585 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a1cdff6e1204 (Metraipolis archive dc_a1cdff6e1204, sha256 8fc9acef8d16761b44b447364ef682d26a46c466c73e4b6070bb6d9f9a9b0ceb)
BibTeX
@misc{dca1cdff6e1204,
title = {mKernel: Fast Multi-GPU, Multi-Node Fused Kernels — arXiv:2609.13585},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.13585},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a1cdff6e1204; sha256 8fc9acef8d16761b44b447364ef682d26a46c466c73e4b6070bb6d9f9a9b0ceb},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.13585. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Communication has become a bottleneck in distributed training and inference of large models. Overlapping communication with computation at the granularity of kernels, on separate streams, reduces only part of this communication cost. Fused kernels often have better performance by transmitting each output tile as soon as it is produced, but existing fused kernels are largely confined to a single NVLink domain. We present mKernel, a library of multi-GPU, multi-node fused kernels that overlap computation, intra-node NVLink communication, and inter-node RDMA at tile granularity. mKernel partitions the streaming multiprocessors (SMs) of a persistent kernel into compute and communication roles, and an on-GPU controller tunes the SM partition adaptively at run time, since the best SM partition varies with the kernel and the input shape. It structures data movement hierarchically so that data traversing the inter-node network is minimized. Finally, it drives the network from the GPU through a lightweight command queue and host proxy implemented directly on RDMA verbs, which allows the same kernels to run on any network backend (e.g. InfiniBand and on AWS EFA); we observe, surprisingly, that GPUDirect Async (IBGDA) yields little additional benefit over host-assisted GPU-initiated communication. We implement five kernels spanning tensor, sequence, and expert parallelism. On two 16-GPU H200 clusters, mKernel achieves speedups of up to 1.72x on GEMM+AllReduce and $1.88\times$ on Ring Attention. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.AI). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Networking and Internet Architecture (cs.NI). Pagina del lavoro: https://arxiv.org/abs/2609.13585 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:29:03.524Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).