Metraipolis › archive › dc_1f4e754147ad
MonoMoE: An Efficient Fused Mega-kernel for Quantized MoE Decoding — arXiv:2609.04244
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.04244
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T22:42:12.848Z
- SHA-256 of the archived text
6febfff4f202e35983f2a13c8b955d0ee2fa6a4247c633c3375d0786daacaecd- Stable URL of this page
https://metraipolis.com/documento/dc_1f4e754147ad
How to cite
MonoMoE: An Efficient Fused Mega-kernel for Quantized MoE Decoding — arXiv:2609.04244 — arXiv (en), https://arxiv.org/abs/2609.04244 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1f4e754147ad (Metraipolis archive dc_1f4e754147ad, sha256 6febfff4f202e35983f2a13c8b955d0ee2fa6a4247c633c3375d0786daacaecd)
BibTeX
@misc{dc1f4e754147ad,
title = {MonoMoE: An Efficient Fused Mega-kernel for Quantized MoE Decoding — arXiv:2609.04244},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.04244},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1f4e754147ad; sha256 6febfff4f202e35983f2a13c8b955d0ee2fa6a4247c633c3375d0786daacaecd},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04244. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Mixture-of-Experts (MoE) layers increase model capacity without proportionally increasing arithmetic, but their sparse expert computation is difficult to execute efficiently during autoregressive decode. Existing grouped and batched GEMMs are token-major: they construct expert-local token tiles and obtain parallelism from the token dimension. When few tokens reach each expert, this organization incurs tile padding and preprocessing, launches short-lived grids that underutilize memory bandwidth, and exposes quantization, activation, and reduction as separate stages. We present \textbf{MonoMoE}, a weight-major persistent megakernel for block-wise quantized MoE decode. MonoMoE places the complete decode-step token tile on the fine-grained tensor-core $N$ dimension and partitions CTAs over expert-weight tiles, eliminating expert-local token materialization and reducing padded arithmetic. A persistent grid fuses routing, top-$k$ selection, quantization, both expert projections, activation, and reduction in one launch; warp specialization and readiness flags overlap auxiliary work with the dominant expert-weight stream. MonoMoE is integrated with vLLM and supports multiple model shapes through generated kernel specializations and offline schedule tuning. On NVIDIA H200 GPUs, MonoMoE accelerates the complete routed-MoE operator by up to $\mathbf{1.54\times}$ over vLLM Triton Grouped GEMM, is $\mathbf{2.20}$--$\mathbf{3.84\times}$ faster than FlashMoE-FP8 adaptation across various models and batch sizes, and reduces end-to-end time per output token by up to $\mathbf{18.7\%}$ across the evaluated FP8 models, while preserving task accuracy. The MonoMoE implementation and supporting artifacts are open source and available in the \href{ this https URL }{FlashInfer repository}.
Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-07 in cs.AR).
Categorie arXiv: Hardware Architecture (cs.AR).
Pagina del lavoro: https://arxiv.org/abs/2609.04244 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T22:26:48.903Z dalla pagina https://arxiv.org/catchup/cs.AR/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).