Metraipolis › archive › dc_bf808a162eb8

Explaining the Roles of Experts in Mixture-of-Experts LLMs for Personalization — arXiv:2610.22127

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.22127
License (as archived)
CC0-1.0
Archived
2026-10-16T09:31:01.781Z
SHA-256 of the archived text
989aa729cc647a52229f4e13721e0a702ac292712736f96bb043e76f82b2ac4c
Stable URL of this page
https://metraipolis.com/documento/dc_bf808a162eb8

How to cite

Explaining the Roles of Experts in Mixture-of-Experts LLMs for Personalization — arXiv:2610.22127 — arXiv (en), https://arxiv.org/abs/2610.22127 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bf808a162eb8 (Metraipolis archive dc_bf808a162eb8, sha256 989aa729cc647a52229f4e13721e0a702ac292712736f96bb043e76f82b2ac4c)
BibTeX
@misc{dcbf808a162eb8,
  title = {Explaining the Roles of Experts in Mixture-of-Experts LLMs for Personalization — arXiv:2610.22127},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.22127},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bf808a162eb8; sha256 989aa729cc647a52229f4e13721e0a702ac292712736f96bb043e76f82b2ac4c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22127. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Personalization of Large Language Models (LLMs) is increasingly important for user-centric applications, yet prior methods mainly adapt dense LLMs, whose polysemantic neurons, each encoding multiple unrelated concepts, hinder localizing the components behind personalization, and thus understanding and controlling it. We propose to personalize Mixture-of-Experts (MoE) LLMs, which sparsely route each token to a few specialized experts, reducing polysemanticity and enabling such localization. We adapt three open-weight MoE LLMs on a large-scale personalization dataset and train crosscoders on paired activations of the personalized and non-personalized models to learn interpretable personalized latents, from which we identify the experts encoding distinct personalization features. The personalized MoE LLMs perform competitively with dense LLMs while activating only a fraction of their parameters, exhibit lower polysemanticity than dense LLMs with similar active parameters, and, within the same model family, yield personalized latents that are less shared with the non-personalized model. Causal interventions show that these latents suffice to recover the personalized model's behavior and that boosting their associated experts moves the non-personalized model toward it, revealing distinct routing paths for different personalization features.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.22127 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:44:18.029Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).