Metraipolis › archive › dc_7a002008f1fe
A Per-Step Load-Balancing Engine for Expert-Parallel MoE Serving — arXiv:2611.11402
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.11402
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-10T18:45:57.932Z
- SHA-256 of the archived text
ce02203597f577203675332cb6fe3bc17a132317bc41bb58d738734124a2ee02- Stable URL of this page
https://metraipolis.com/documento/dc_7a002008f1fe
How to cite
A Per-Step Load-Balancing Engine for Expert-Parallel MoE Serving — arXiv:2611.11402 — arXiv (en), https://arxiv.org/abs/2611.11402 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7a002008f1fe (Metraipolis archive dc_7a002008f1fe, sha256 ce02203597f577203675332cb6fe3bc17a132317bc41bb58d738734124a2ee02)
BibTeX
@misc{dc7a002008f1fe,
title = {A Per-Step Load-Balancing Engine for Expert-Parallel MoE Serving — arXiv:2611.11402},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.11402},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7a002008f1fe; sha256 ce02203597f577203675332cb6fe3bc17a132317bc41bb58d738734124a2ee02},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.11402. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
In expert-parallel Mixture-of-Experts serving, a layer finishes only when the most loaded GPU of its group does. Token routing shifts at every decode step and concentrates on a few experts, so most of the group's reserved GPU time waits on stragglers. Recovering that time means re-deciding, at every layer of every decode step, how each replicated expert's tokens divide between its copies. The exact linear-program solvers that make this decision optimally take longer than many of the layers they serve, and their working set outgrows GPU shared memory at deployment scale. We introduce Relief, a bidirectional pairwise relaxation that makes the linear program unnecessary, coming within a fraction of a point of the per-step optimum in a few passes of integer halving over the token counts. We build and measure three deployments, CUDA kernels on the serving GPU, the same verified engine on an FPGA over PCIe, and a fixed-function block beside the accelerator. On decode routing from three open MoE families, Relief holds the balance near the per-step optimum at every group size and layer time measured. Scored as effective utilization relative to that optimum, the kernel alone matches or exceeds every deployable scheme at every group size, from the hundred-microsecond layer on. The on-die engine widens that lead at every point, recovering up to 37.3 percentage points over the deployed static placement, while adding 0.02% of a current accelerator's transistors. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno). Categorie arXiv: Hardware Architecture (cs.AR) ; Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2611.11402 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-10T18:38:07.452Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).