Metraipolis › archive › dc_122e72b4c7ef
Scaling Multilingual Mixture-of-Experts: How Language Coverage Changes Capacity and Compute Allocation — arXiv:2610.20247
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.20247
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T14:23:46.782Z
- SHA-256 of the archived text
6f039c925f098a9c091d0f71850c0c00c4928ea4090602e1b6ab596d05a347a6- Stable URL of this page
https://metraipolis.com/documento/dc_122e72b4c7ef
How to cite
Scaling Multilingual Mixture-of-Experts: How Language Coverage Changes Capacity and Compute Allocation — arXiv:2610.20247 — arXiv (en), https://arxiv.org/abs/2610.20247 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_122e72b4c7ef (Metraipolis archive dc_122e72b4c7ef, sha256 6f039c925f098a9c091d0f71850c0c00c4928ea4090602e1b6ab596d05a347a6)
BibTeX
@misc{dc122e72b4c7ef,
title = {Scaling Multilingual Mixture-of-Experts: How Language Coverage Changes Capacity and Compute Allocation — arXiv:2610.20247},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.20247},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_122e72b4c7ef; sha256 6f039c925f098a9c091d0f71850c0c00c4928ea4090602e1b6ab596d05a347a6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.20247. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multilingual models share capacity across languages, so adding languages can hurt performance on existing ones. Mixture-of-experts models expand this capacity with little additional per-token computation. Given this extra capacity, does adding experts reduce what each additional language costs? To answer this question, we propose a repetition-aware joint scaling law relating number of active parameters, training tokens per language, number of experts, and language coverage. Fitted on 26,300 per-language loss observations from 64 pretrained configurations, our law reduces prediction error on held-out configurations by 28% relative to the strongest evaluated baseline. At matched number of active parameters and training tokens per language, 32 experts reduce the loss penalty of expanding from 12 to 200 languages by 83% compared with a dense model. Our law further shows that broader language coverage favors more experts across a wider range of compute budgets, guiding capacity allocation as multilingual models scale. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.20247 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T13:53:35.944Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).