Metraipolis › archive › dc_32043b9afe8b

Characterizing Recoverable Expert-Utilization Signatures in Fine-Tuned Mixture-of-Experts LLMs — arXiv:2610.41039

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41039
License (as archived): CC0-1.0
Archived: 2026-11-03T07:42:36.522Z
SHA-256 of the archived text: c5d78349ccb56ff700d9238fccf36f179f10dba2a0da785e2a74843c6bfd2c88
Stable URL of this page: https://metraipolis.com/documento/dc_32043b9afe8b

How to cite

Characterizing Recoverable Expert-Utilization Signatures in Fine-Tuned Mixture-of-Experts LLMs — arXiv:2610.41039 — arXiv (en), https://arxiv.org/abs/2610.41039 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_32043b9afe8b (Metraipolis archive dc_32043b9afe8b, sha256 c5d78349ccb56ff700d9238fccf36f179f10dba2a0da785e2a74843c6bfd2c88)

BibTeX

@misc{dc32043b9afe8b,
  title = {Characterizing Recoverable Expert-Utilization Signatures in Fine-Tuned Mixture-of-Experts LLMs — arXiv:2610.41039},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41039},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_32043b9afe8b; sha256 c5d78349ccb56ff700d9238fccf36f179f10dba2a0da785e2a74843c6bfd2c88},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41039. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As Mixture-of-Experts (MoE) large language models are increasingly deployed in distributed environments, from federated learning to edge-cloud systems, their architecture harbors an underexplored landscape of security vulnerabilities. Moving beyond textual analysis, we uncover structural memorization: a novel vulnerability where a fine-tuned model memorizes not only its training data but also the internal pathway used to process it, creating a characteristic routing fingerprint unique to the training process. We provide comprehensive empirical evidence that structural memorization emerges consistently even from a few samples, and that its intensity is directly influenced by the choice of trainable modules. We show that these memorized expert pathways can be stealthily recovered through the model's unconditional generation, using only routing and output logits, with no knowledge of the fine-tuning data or procedure. In distributed deployments, this vulnerability is amplified as routing information is exposed at multiple trust boundaries, including federated aggregation points, edge-cloud interfaces, and multi-tenant monitoring systems. Recovery exposes a critical blind spot: traditional membership inference metrics, viewed through the lens of expert routing, fail to identify memorized data corresponding to these structural pathways. To close this gap, we devise composite metrics combining reference model tail likelihood with complexity-aware normalization, significantly improving detection of memorized routing structures. This enables adversaries to pinpoint critical experts shaped by fine-tuning, whether dominant or dormant, and execute targeted attacks such as router manipulation or backdoor injection in realistic distributed settings where attackers lack training data but can observe routing patterns across system components.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2610.41039 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T07:39:37.358Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).