Metraipolis › archive › dc_a65f685ff815
MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Models — arXiv:2609.33893
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.33893
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T22:37:50.722Z
- SHA-256 of the archived text
f1748ec9db66aa8156f4ba1d8274fb55c40854af2ef5cb1964f28cdd62a125ba- Stable URL of this page
https://metraipolis.com/documento/dc_a65f685ff815
How to cite
MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Models — arXiv:2609.33893 — arXiv (en), https://arxiv.org/abs/2609.33893 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a65f685ff815 (Metraipolis archive dc_a65f685ff815, sha256 f1748ec9db66aa8156f4ba1d8274fb55c40854af2ef5cb1964f28cdd62a125ba)
BibTeX
@misc{dca65f685ff815,
title = {MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Models — arXiv:2609.33893},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.33893},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a65f685ff815; sha256 f1748ec9db66aa8156f4ba1d8274fb55c40854af2ef5cb1964f28cdd62a125ba},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33893. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large audio-language models (LALMs) produce fluent responses about audio but often hallucinate by making plausible yet ungrounded claims. Existing audio hallucination benchmarks mainly measure response correctness, leaving it unclear whether an LALM hallucinates or simply fails to understand the audio. We challenge correctness-based evaluation by defining two hallucination categories: (i) context, where claims are not grounded in the audio; and (ii) knowledge, where claims about audio-related topics lack support from externally verifiable facts. We introduce MISHAP-Bench, a comprehensive benchmark with 12,000 challenging open-ended question-audio pairs and a rigorous evaluation pipeline covering both categories. To evaluate open-ended responses, we propose a groundedness judge that uses reference rubrics and judge prompts guided by human annotations. We extensively evaluate ten state-of-the-art LALMs and show that hallucination remains substantial. Even a frontier model such as Gemini 3.7 Flash reaches a hallucination rate of 36.5%. We further adapt and benchmark four mitigation methods from multiple domains for LALMs. Despite some improvements, effective hallucination mitigation remains an open challenge. Finally, we call on the community to evaluate hallucination and benchmark mitigation methods with MISHAP-Bench. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.33893 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T22:14:28.057Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).