Metraipolis › archive › dc_ce3eaafe662c
DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding — arXiv:2608.14385
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14385
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T02:21:31.543Z
- SHA-256 of the archived text
c4585a9b618067e21b5a068df3b6a64a0d15a9678a075655b8bcbbb9c464dc68- Stable URL of this page
https://metraipolis.com/documento/dc_ce3eaafe662c
How to cite
DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding — arXiv:2608.14385 — arXiv (en), https://arxiv.org/abs/2608.14385 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ce3eaafe662c (Metraipolis archive dc_ce3eaafe662c, sha256 c4585a9b618067e21b5a068df3b6a64a0d15a9678a075655b8bcbbb9c464dc68)
BibTeX
@misc{dcce3eaafe662c,
title = {DeaMoE: Efficient MoE Structure for Fast Small-Batch Decoding — arXiv:2608.14385},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14385},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ce3eaafe662c; sha256 c4585a9b618067e21b5a068df3b6a64a0d15a9678a075655b8bcbbb9c464dc68},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14385. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Mixture-of-Experts (MoE) models have been widely adopted in real-time interactive applications such as coding assistants, real-time audio-video interaction systems. To meet the extremely low response latency requirements of these scenarios, practitioners commonly employ small-batch decoding, under which MoE inference becomes memory-bound and is severely bottlenecked by expert weight loading. However, this bottleneck has received limited attention, and existing solutions such as post-training weight compression or fine-grained expert design during pre-training either degrade model accuracy or introduce additional computation and communication overhead. To tackle this issue, we propose DeaMoE, a decoding-efficient MoE architecture, in which the experts are grouped into several departments, and the experts belonging to the same department share most parameters since they come from the same professional field, and additionally each expert contains a few private parameters to reflect its uniqueness. Moreover, we design customized two-stage routing strategy for DeaMoE to avoid redundant loading, under which DeaMoE greatly improves the efficiency during LLM decoding. Compared with vanilla MoE, DeaMoE reduces per-step loaded weights by up to 50.9% and achieves up to 1.33 end-to-end TPOT speedup for the pre-trained 7B model on A40, and up to 2.00x and 1.97x peak speedup for DeepSeek-V3 on A40 and H100 in microbenchmarks. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-17 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.14385 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T02:02:02.940Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).