Metraipolis › archive › dc_d4cdf56c2b06
BigMoMo: Efficient Inference of Large-Scale MoE with Speculative Decoding on Mobile Devices — arXiv:2609.14643
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.14643
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T23:17:47.238Z
- SHA-256 of the archived text
e3915076ff327d43e43915fe91626597cae06f7c05e17ad26ed1ede4415e0abb- Stable URL of this page
https://metraipolis.com/documento/dc_d4cdf56c2b06
How to cite
BigMoMo: Efficient Inference of Large-Scale MoE with Speculative Decoding on Mobile Devices — arXiv:2609.14643 — arXiv (en), https://arxiv.org/abs/2609.14643 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d4cdf56c2b06 (Metraipolis archive dc_d4cdf56c2b06, sha256 e3915076ff327d43e43915fe91626597cae06f7c05e17ad26ed1ede4415e0abb)
BibTeX
@misc{dcd4cdf56c2b06,
title = {BigMoMo: Efficient Inference of Large-Scale MoE with Speculative Decoding on Mobile Devices — arXiv:2609.14643},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.14643},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d4cdf56c2b06; sha256 e3915076ff327d43e43915fe91626597cae06f7c05e17ad26ed1ede4415e0abb},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.14643. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Mixture-of-Experts (MoE) models expand language model capacity on smartphones, but expert offloading remains constrained by limited DRAM capacity and costly data movement. Sequential token routing couples expert execution to fragmented flash reads and multistage NPU preparation, leaving sparse computation stalled on weight transfers. Each transfer serves few tokens before execution moves on. We exploit the multi-token verification window of speculative decoding to decouple expert movement from single-token execution, enabling weight reuse, contiguous flash reads, and load-compute overlap. We present \textsc{BigMoMo}, a mobile MoE runtime that exploits this window across the memory hierarchy. It prunes speculative branches and expert activations using acceptance rates, routing impact, and movement cost; reorganizes on-flash experts according to runtime co-loading patterns; and batches ready experts to overlap NPU computation with pending transfers. Across four MoE models and five benchmarks on two mobile platforms, \textsc{BigMoMo} achieves mean decoding speedups of $4.83\times$ over on-demand autoregressive offloading and $1.82\times$ over the best speculative MoE baseline, supporting MoE models up to 80B parameter.
Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-15 in cs.DC).
Categorie arXiv: Hardware Architecture (cs.AR) ; Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2609.14643 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T22:54:13.016Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).