Metraipolis › archive › dc_6658bf4d08ec

HDA-MoE: Hybrid Parallelism and Dynamic, Adaptive Scheduling for Mixture-of-Experts with 3D Near-Memory Processing — arXiv:2609.08682

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.08682
License (as archived)
CC0-1.0
Archived
2026-10-16T13:14:41.727Z
SHA-256 of the archived text
a7a1187e48aecda9c2a7d1305fc4d97ecda45a89ff7666428e6d0daac8edd6f9
Stable URL of this page
https://metraipolis.com/documento/dc_6658bf4d08ec

How to cite

HDA-MoE: Hybrid Parallelism and Dynamic, Adaptive Scheduling for Mixture-of-Experts with 3D Near-Memory Processing — arXiv:2609.08682 — arXiv (en), https://arxiv.org/abs/2609.08682 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6658bf4d08ec (Metraipolis archive dc_6658bf4d08ec, sha256 a7a1187e48aecda9c2a7d1305fc4d97ecda45a89ff7666428e6d0daac8edd6f9)
BibTeX
@misc{dc6658bf4d08ec,
  title = {HDA-MoE: Hybrid Parallelism and Dynamic, Adaptive Scheduling for Mixture-of-Experts with 3D Near-Memory Processing — arXiv:2609.08682},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.08682},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6658bf4d08ec; sha256 a7a1187e48aecda9c2a7d1305fc4d97ecda45a89ff7666428e6d0daac8edd6f9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.08682. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Mixture-of-Experts (MoE) architectures have become a key technique for scaling Large Language Models (LLMs), enabling high model capacity with reduced computational cost. However, this efficiency comes at the expense of increased memory capacity and bandwidth demands. Recent 3D Near-Memory Processing (NMP) architectures, which vertically integrate memory and compute through hybrid bonding, provide high internal bandwidth and energy efficiency, making them attractive for accelerating MoE inference. Nevertheless, the distributed memory and compute organization of NMP systems introduces new challenges for mapping MoE workloads. Existing parallelization strategies, such as Tensor Parallelism (TP) and Expert Parallelism (EP), suffer from either high communication costs or unbalanced computation utilization, leading to inferior efficiency. In addition, the dynamic routing behavior of MoE models further complicates efficient deployment. To address these challenges, we present HDA-MoE, a framework that optimizes MoE execution on NMP architectures through hybrid parallel deployment and runtime scheduling. HDA-MoE integrates an offline hybrid parallel mapping algorithm with an online dynamic and adaptive scheduling mechanism to reduce communication overhead while improving computation utilization. Experimental results show that HDA-MoE achieves a speedup of 1.1x--3.4x over TP, 1.1x--1.5x over EP, 1.1x--3.7x over the Hybrid TP-EP compute-balanced baseline, and 1.1x--1.3x over HD-MoE. Source code is available at this https URL .

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-09 in cs.AR).
Categorie arXiv: Hardware Architecture (cs.AR).
Pagina del lavoro: https://arxiv.org/abs/2609.08682 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T13:02:36.891Z dalla pagina https://arxiv.org/catchup/cs.AR/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).