Metraipolis › archive › dc_6265be976c2e
Audio LLMs Know When They Can't Hear You — arXiv:2609.30625
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.30625
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T01:38:15.600Z
- SHA-256 of the archived text
b4d0e3158826b0b1cb2d480a95281661fd756f9683b59e4955f7180814067085- Stable URL of this page
https://metraipolis.com/documento/dc_6265be976c2e
How to cite
Audio LLMs Know When They Can't Hear You — arXiv:2609.30625 — arXiv (en), https://arxiv.org/abs/2609.30625 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6265be976c2e (Metraipolis archive dc_6265be976c2e, sha256 b4d0e3158826b0b1cb2d480a95281661fd756f9683b59e4955f7180814067085)
BibTeX
@misc{dc6265be976c2e,
title = {Audio LLMs Know When They Can't Hear You — arXiv:2609.30625},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.30625},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6265be976c2e; sha256 b4d0e3158826b0b1cb2d480a95281661fd756f9683b59e4955f7180814067085},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30625. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Audio large language models allow users to interact with the model through speech. When an input recording is too degraded, the model may misinterpret the user's query and respond based on an incorrect transcription. In this paper, we study model-conditional transcription reliability: whether an Audio LLM can recognize when its own transcription is unreliable. We first prompt the Audio LLM to assess whether its own transcription would be reliable, and find that the model is a poor judge of its own transcription reliability: in most cases, it predicts that its transcription will be reliable. We find that existing approaches, including speech quality predictors, audio LLM generation uncertainty, and transcript-conditioned WER estimation, provide limited signals for detecting transcription failures. In contrast, we discover that transcription reliability is strongly represented in the model's audio-encoder representations. Based on this observation, we devise a lightweight reliability predictor that operates on representations extracted by the frozen audio encoder and predicts the reliability class before generation. The reliability predictor can trigger a clarification request from the user when their voice query is predicted to be unreliable, while allowing reliable queries to proceed without modifying the underlying Audio LLM. Our predictor achieves 81.10% in-domain and 78.09% cross-domain macro-F1 scores, outperforming the strongest baselines by 10.33 and 11.93 points, respectively. Finally, we show that reliability labels can transfer across Audio LLM families, and that transfer performance is closely related to the alignment of their model-specific reliability boundaries. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.30625 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T01:12:14.258Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).