Metraipolis › archive › dc_d49fc3853caa

Zepp: Accelerating Distributed MoE Serving under Relaxed Balance Constraints — arXiv:2610.11158

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.11158
License (as archived)
CC0-1.0
Archived
2026-10-09T06:13:55.793Z
SHA-256 of the archived text
807844c04f949f1ea889dc7273c1181f93d1d5a4173197c012ee832f5486c6c4
Stable URL of this page
https://metraipolis.com/documento/dc_d49fc3853caa

How to cite

Zepp: Accelerating Distributed MoE Serving under Relaxed Balance Constraints — arXiv:2610.11158 — arXiv (en), https://arxiv.org/abs/2610.11158 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d49fc3853caa (Metraipolis archive dc_d49fc3853caa, sha256 807844c04f949f1ea889dc7273c1181f93d1d5a4173197c012ee832f5486c6c4)
BibTeX
@misc{dcd49fc3853caa,
  title = {Zepp: Accelerating Distributed MoE Serving under Relaxed Balance Constraints — arXiv:2610.11158},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.11158},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d49fc3853caa; sha256 807844c04f949f1ea889dc7273c1181f93d1d5a4173197c012ee832f5486c6c4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.11158. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As Mixture-of-Experts (MoE) models continue to scale, serving them increasingly relies on expert parallelism (EP) across a growing number of devices. Yet skewed expert workloads create imbalance across computation, communication, and memory, making load balancing a central optimization objective in distributed MoE serving. We observe that balance is not free: operations introduced to balance one dimension can themselves be expensive or imbalanced. This motivates us to rethink balance as a constraint rather than an optimization objective. We present Zepp, which directly optimizes the bottleneck communication in distributed MoE serving subject to simplified balance constraints on physical resources, i.e., GPUs and NICs. Zepp progressively optimizes inter-node communication across placement, routing, and execution. It first places expert replicas to reduce token communication under GPU constraints, then reshapes communication flows through split and merge primitives under NIC constraints, and finally partitions and schedules ex- pert computation to overlap the resulting communication. To adapt to dynamic workloads, Zepp jointly coordinates computation, token communication, and expert-weight movement at each iteration. Together, these designs allow Zepp to pursue the most efficient execution rather than a single-dimension balanced one. We implement Zepp and evaluate it against 7 state-of-the-art MoE serving systems, achieving up to 6.68$\times$ MoE layer speedup and a geometric mean speedup of 1.86$\times$ over the fastest competing baseline.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2610.11158 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:47:30.192Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).