Metraipolis › archive › dc_524dd9783f2f
CascadeEP: Asynchronous Expert Execution for MoE Prefill under Attention Imbalance — arXiv:2609.33252
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.33252
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T22:12:06.007Z
- SHA-256 of the archived text
f83a34fbbb6528938b230faccf5c5206742696c2e29a21e1275828307c10acce- Stable URL of this page
https://metraipolis.com/documento/dc_524dd9783f2f
How to cite
CascadeEP: Asynchronous Expert Execution for MoE Prefill under Attention Imbalance — arXiv:2609.33252 — arXiv (en), https://arxiv.org/abs/2609.33252 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_524dd9783f2f (Metraipolis archive dc_524dd9783f2f, sha256 f83a34fbbb6528938b230faccf5c5206742696c2e29a21e1275828307c10acce)
BibTeX
@misc{dc524dd9783f2f,
title = {CascadeEP: Asynchronous Expert Execution for MoE Prefill under Attention Imbalance — arXiv:2609.33252},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.33252},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_524dd9783f2f; sha256 f83a34fbbb6528938b230faccf5c5206742696c2e29a21e1275828307c10acce},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33252. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Mixture-of-experts (MoE) serving commonly deploys data and expert parallelism (DEP): attention replicas run distinct request batches while routed experts are sharded across an expert-parallel (EP) group. During prefill, attention replicas finish dispatch at different times, but synchronous EP delays expert feed-forward network (FFN) computation until routed inputs from all replicas are ready. Request schedulers seek to balance load while reusing the key-value (KV) cache of shared prompt prefixes to avoid redundant prefill computation. These goals can conflict when a replica holding a matching prefix is already overloaded, leaving residual attention imbalance. We present ASYNCEP, a distributed execution engine for MoE prefill. ASYNCEP proposes three mechanisms. Asynchronous EP allows expert computation to start before tokens from all attention replicas are ready. streamFFN batches ready tokens to balance early execution with FFN computation efficiency. Opportunistic expert weight fetching (OEWF) allows a faster replica to fetch expert weights and execute unstarted work from other GPUs. We evaluate ASYNCEP on DeepSeek-V4-Flash, DeepSeek-V4-Pro, and GLM-5.3, and our results show that ASYNCEP achieves up to 1.48x speedup in p95 time-to-first-token (TTFT) and improves the inference throughput by up to 1.17x. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-29 in cs.DC). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2609.33252 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T21:53:13.674Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).