Metraipolis › archive › dc_9caeb66642f2

Refusal Is Not Robustness: Auditing Confident Fabrication in Large Language Models on a Provably Uninformative Clinical Pain Speech Transcript — arXiv:2608.26167

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.26167
License (as archived)
CC0-1.0
Archived
2026-10-17T14:16:13.982Z
SHA-256 of the archived text
354bf73e48d0cadf417f6f8b62e708112fd3d281da74cdeb43cb59a97c2333ab
Stable URL of this page
https://metraipolis.com/documento/dc_9caeb66642f2

How to cite

Refusal Is Not Robustness: Auditing Confident Fabrication in Large Language Models on a Provably Uninformative Clinical Pain Speech Transcript — arXiv:2608.26167 — arXiv (en), https://arxiv.org/abs/2608.26167 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9caeb66642f2 (Metraipolis archive dc_9caeb66642f2, sha256 354bf73e48d0cadf417f6f8b62e708112fd3d281da74cdeb43cb59a97c2333ab)
BibTeX
@misc{dc9caeb66642f2,
  title = {Refusal Is Not Robustness: Auditing Confident Fabrication in Large Language Models on a Provably Uninformative Clinical Pain Speech Transcript — arXiv:2608.26167},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.26167},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9caeb66642f2; sha256 354bf73e48d0cadf417f6f8b62e708112fd3d281da74cdeb43cb59a97c2333ab},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.26167. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Hallucination and abstention benchmarks rarely establish that a model could not have known the correct answer, making it difficult to distinguish appropriate abstention from an unsupported prediction. Seven large language models were evaluated on the TAME Pain speech corpus. Participants read phonetically balanced Harvard Sentences while one hand was immersed in cold or warm water and reported pain only during periodic pain statements. This protocol generated 5,750 no signal Harvard Sentence utterances whose transcripts contained no lexical pain information and 1,294 signal pain statement utterances in which the pain rating was explicitly spoken. In the no signal arm, pain was recoverable from acoustic features (AUC 0.622, 95% CI 0.553 to 0.662), whereas transcript based prediction was near chance (AUC 0.489, 95% CI 0.418 to 0.504). Because automatic speech recognition removes the acoustic pain cues, any pain score inferred solely from the transcript is unsupported by the available evidence. Under cooperative prompting, six models abstained on nearly all no signal transcripts, correctly extracted spoken pain ratings in the positive control task with accuracies ranging from 0.939 to 1.00, and maintained an expected calibration error of at most 0.100. Under authority framed prompts, abstention became prompt dependent, with the same model ranging from 0.18 to 1.00 across equivalent prompt phrasings. Most models produced low confidence estimates when forced to answer, whereas Gemini 2.5 Flash and Llama 3.1 8B consistently generated confident pain scores with confident fabrication rates of 0.53 and 0.76, compared with at most 0.15 for all other models. No significant demographic effects were observed in forced responses, with all $p$ values greater than or equal to 0.20.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Emerging Technologies (cs.ET); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS).
Pagina del lavoro: https://arxiv.org/abs/2608.26167 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T13:16:53.441Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).