Metraipolis › archive › dc_5d148ec76bbb
Do Audio Language Models Use Paralinguistic Evidence? Counterfactual Audits for Response Evaluation — arXiv:2608.06718
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.06718
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T09:46:48.058Z
- SHA-256 of the archived text
9b4d2c93f1af6be7498cd48d54b6c24a6a8db57eeba49ad330aeae66c454c02f- Stable URL of this page
https://metraipolis.com/documento/dc_5d148ec76bbb
How to cite
Do Audio Language Models Use Paralinguistic Evidence? Counterfactual Audits for Response Evaluation — arXiv:2608.06718 — arXiv (en), https://arxiv.org/abs/2608.06718 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5d148ec76bbb (Metraipolis archive dc_5d148ec76bbb, sha256 9b4d2c93f1af6be7498cd48d54b6c24a6a8db57eeba49ad330aeae66c454c02f)
BibTeX
@misc{dc5d148ec76bbb,
title = {Do Audio Language Models Use Paralinguistic Evidence? Counterfactual Audits for Response Evaluation — arXiv:2608.06718},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.06718},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5d148ec76bbb; sha256 9b4d2c93f1af6be7498cd48d54b6c24a6a8db57eeba49ad330aeae66c454c02f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.06718. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Audio-language models (ALMs) are increasingly used as judges for speech-to-speech systems, but a judge that receives audio may not actually use paralinguistic evidence. We introduce counterfactual audits for paralinguistic response evaluation. Each audit item holds the transcript fixed while varying affect, prosody, or the timing of an affective shift, forcing a valid judge to track the audio cue rather than lexical content or response style. We evaluate ALM judges using a native one-context judgment protocol and a contrastive recoverability control, then further decompose each item into its constituent perception and response-mapping skills. This yields useful diagnostic states that identify different sources of judge failures. Across Gemini, GPT, and open audio models, we find that contrastive success often overstates native judge reliability, and that similar aggregate accuracies can hide different failure modes. These results suggest that ALM judges should not be evaluated by accuracy alone, instead requiring thorough behavioral audits before deployment. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-10 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.06718 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T09:46:37.214Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).