Metraipolis › archive › dc_02eedcf79dcc

Weave: Fine-Grained Dynamic SM Scheduling in an MoE Megakernel for Compute-Communication Overlap — arXiv:2609.21483

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.21483
License (as archived): CC0-1.0
Archived: 2026-10-10T01:37:59.936Z
SHA-256 of the archived text: e43b8db6e971d619ce66590b0b57e915641c5a162e5dedd26e0de6d505712843
Stable URL of this page: https://metraipolis.com/documento/dc_02eedcf79dcc

How to cite

Weave: Fine-Grained Dynamic SM Scheduling in an MoE Megakernel for Compute-Communication Overlap — arXiv:2609.21483 — arXiv (en), https://arxiv.org/abs/2609.21483 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_02eedcf79dcc (Metraipolis archive dc_02eedcf79dcc, sha256 e43b8db6e971d619ce66590b0b57e915641c5a162e5dedd26e0de6d505712843)

BibTeX

@misc{dc02eedcf79dcc,
  title = {Weave: Fine-Grained Dynamic SM Scheduling in an MoE Megakernel for Compute-Communication Overlap — arXiv:2609.21483},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.21483},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_02eedcf79dcc; sha256 e43b8db6e971d619ce66590b0b57e915641c5a162e5dedd26e0de6d505712843},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.21483. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Mixture-of-Experts (MoE) inference under expert parallelism (EP) turns each MoE layer into a distributed computation with costly dispatch and combine communication. State-of-the-art systems reduce this cost through communication-computation overlap, splitting the GPU's SMs for communication and computation respectively. However, this approach still leaves GPU resources wasted along two dimensions. Spatially, the best SM split is determined by each layer's routing result and varies across layers and GPUs, so fixed policies mismatch the workload and waste either NVLink bandwidth or compute throughput. Temporally, complex MoE data dependencies introduce bubbles that leave SMs idle. We present Weave, to our knowledge the first MoE overlap system that performs fine-grained dynamic SM scheduling - deciding per layer and per GPU by routing results at runtime. Once routing completes, each layer's communication and computation volumes become known; Weave exploits this predictability through a lightweight cost model running inside the persistent megakernel: a spatial scheduler partitions SMs into communication workers and computation workers to match the communication/computation throughput ratio, and a temporal scheduler coordinates the two worker groups to minimize SM idleness. On 4x H100 SXM GPUs across six mainstream MoE models, Weave achieves a 2.89x geometric-mean MoE-layer speedup and a 1.33x geometric-mean end-to-end speedup over five state-of-the-art baselines.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-21 in cs.DC).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2609.21483 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T01:05:30.962Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).