Metraipolis › archive › dc_3df00879d741
Structure Aware Neural Architecture Search for Mixture of Experts — arXiv:2608.29817
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.29817
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T08:36:55.379Z
- SHA-256 of the archived text
557d41e69f810c11e6cb48dbc2f22c39b76c4f97c063e1b5ce6f51dba74f01b8- Stable URL of this page
https://metraipolis.com/documento/dc_3df00879d741
How to cite
Structure Aware Neural Architecture Search for Mixture of Experts — arXiv:2608.29817 — arXiv (en), https://arxiv.org/abs/2608.29817 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3df00879d741 (Metraipolis archive dc_3df00879d741, sha256 557d41e69f810c11e6cb48dbc2f22c39b76c4f97c063e1b5ce6f51dba74f01b8)
BibTeX
@misc{dc3df00879d741,
title = {Structure Aware Neural Architecture Search for Mixture of Experts — arXiv:2608.29817},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.29817},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3df00879d741; sha256 557d41e69f810c11e6cb48dbc2f22c39b76c4f97c063e1b5ce6f51dba74f01b8},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29817. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Neural Architecture Search (NAS) has so far rarely been applied to Mixture-of-Experts (MoE) models, and existing MoE designs leave the alignment between experts and the structure of the data to emerge on its own. We propose an architecture search framework that makes this alignment an explicit search variable: the assignment of data clusters to experts is optimised jointly with the per-expert architectures. We cast the joint problem as a cluster-aware likelihood maximisation, show that it coincides with the incomplete-data maximum likelihood of a latent-variable mixture, and solve it by a generalised Expectation-Maximisation procedure whose otherwise intractable expert-quality term is supplied by an adaptively refined surrogate. We prove that the iterates converge whenever the surrogate errors are summable, and that at every limit point no candidate the search produces improves the true objective. On a heterogeneous image-classification mixture the method recovers the underlying domain partition on 95% of clusters without ever observing domain labels, and on that benchmark and a four-domain time-series forecasting one alike it outperforms the MoE and NAS baselines that likewise use no label information. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.29817 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T08:13:26.768Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).