Metraipolis › archive › dc_0b77873f167d
ISER-MoE: Dynamic In-Switch Replication of Hot Experts for Load Balancing in Expert-Parallel — arXiv:2611.09008
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.09008
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-09T22:36:30.370Z
- SHA-256 of the archived text
2253b8dcfe8c34eaccfbc7d959a6c09b476c080525e1a9eab5b6cedb0cbbd079- Stable URL of this page
https://metraipolis.com/documento/dc_0b77873f167d
How to cite
ISER-MoE: Dynamic In-Switch Replication of Hot Experts for Load Balancing in Expert-Parallel — arXiv:2611.09008 — arXiv (en), https://arxiv.org/abs/2611.09008 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0b77873f167d (Metraipolis archive dc_0b77873f167d, sha256 2253b8dcfe8c34eaccfbc7d959a6c09b476c080525e1a9eab5b6cedb0cbbd079)
BibTeX
@misc{dc0b77873f167d,
title = {ISER-MoE: Dynamic In-Switch Replication of Hot Experts for Load Balancing in Expert-Parallel — arXiv:2611.09008},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.09008},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0b77873f167d; sha256 2253b8dcfe8c34eaccfbc7d959a6c09b476c080525e1a9eab5b6cedb0cbbd079},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09008. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Expert-parallel mixture-of-experts (MoE) layers run at the pace of the most loaded GPU, and a few hot experts carry most of the excess load. Replicating them at every MoE layer balances the load only if their weights arrive within the layer, and broadcasting from the owner is the bottleneck: point-to-point copies overload it, and baseline broadcasts pay latency or bandwidth overheads. We present ISER-MoE (In-Switch Expert Replication for MoE), built in TensorRT-LLM, which multicasts hot experts through the NVLink switch to fixed GPU groups, so each byte leaves its owner once. A GPU scheduler assigns hot experts to groups per layer, and a multi-port broadcast spreads the injection over the group's ports, both overlapped with the shared expert. In trace replay, the scheduler removed 98-99% of the excess load; in prefill serving of DeepSeek-V4-Pro, throughput rose 17.4% on a math benchmark and 7.2% on agentic prompts. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2611.09008 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-09T22:31:17.043Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).