Metraipolis › archive › dc_939b029630ae
Mixture-of-Expert Blocks Contain Strong Hallucination Detection Signals — arXiv:2608.17687
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.17687
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T14:28:46.979Z
- SHA-256 of the archived text
7007fe79e28e7159706e37f7893c8d66490e09d9f8db31616ead03196560ecf0- Stable URL of this page
https://metraipolis.com/documento/dc_939b029630ae
How to cite
Mixture-of-Expert Blocks Contain Strong Hallucination Detection Signals — arXiv:2608.17687 — arXiv (en), https://arxiv.org/abs/2608.17687 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_939b029630ae (Metraipolis archive dc_939b029630ae, sha256 7007fe79e28e7159706e37f7893c8d66490e09d9f8db31616ead03196560ecf0)
BibTeX
@misc{dc939b029630ae,
title = {Mixture-of-Expert Blocks Contain Strong Hallucination Detection Signals — arXiv:2608.17687},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.17687},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_939b029630ae; sha256 7007fe79e28e7159706e37f7893c8d66490e09d9f8db31616ead03196560ecf0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17687. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Despite their widespread use, Large Language Models (LLMs) remain limited by a fundamental problem: the generation of plausible but false content, known as hallucinations. Most existing detection methods operate at the answer or sentence level, yet per-token detection is essential for localizing hallucinated spans and enabling fine-grained interventions. In this paper, we explore the use of the Mixture-of-Experts (MoE) paradigm to address this gap. In MoE architectures, a single forward pass activates a sparse subset of experts (i.e., distinct feedforward networks per layer) via a routing mechanism, producing internal signals (e.g., router entropy, expert disagreement, and expert usage patterns) that are unavailable in dense architectures and have not been previously exploited for hallucination detection. To this end, we introduce InnerExpert, the first method to leverage these MoE-specific signals for per-token hallucination detection. InnerExpert combines routing-level and standard transformer signals into compact per-token feature vectors, classified by a lightweight detector trained on labels produced by an LLM-as-a-judge pipeline, which enables continuous model updates without manual annotation. Our results show that InnerExpert outperforms existing methods across five datasets and two MoE architectures, achieving up to 0.91 answer-level and 0.76 token-level AUROC, while requiring only a single forward pass. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.17687 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T14:27:54.294Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).