Metraipolis › archive › dc_730679c22c4e

When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information — arXiv:2608.09080

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.09080
License (as archived)
CC0-1.0
Archived
2026-10-19T08:50:14.601Z
SHA-256 of the archived text
e7c5fdf49f46020fb22d45b0ca6219593baa9e51cc81f262f4592c2fd1127b75
Stable URL of this page
https://metraipolis.com/documento/dc_730679c22c4e

How to cite

When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information — arXiv:2608.09080 — arXiv (en), https://arxiv.org/abs/2608.09080 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_730679c22c4e (Metraipolis archive dc_730679c22c4e, sha256 e7c5fdf49f46020fb22d45b0ca6219593baa9e51cc81f262f4592c2fd1127b75)
BibTeX
@misc{dc730679c22c4e,
  title = {When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information — arXiv:2608.09080},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.09080},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_730679c22c4e; sha256 e7c5fdf49f46020fb22d45b0ca6219593baa9e51cc81f262f4592c2fd1127b75},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.09080. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks. However, their reliability under uncertainty remains poorly understood which raises critical concerns for deployment in high-stakes clinical settings. In such environments, incorrect predictions are inherently risky, but confident incorrect predictions can be particularly harmful as they may mislead clinical decision-making. In this paper, we conduct a systematic behavioral analysis of LLMs under clinical information uncertainty. We propose an evaluation framework based on the MedMCQA dataset consisting of two complementary uncertainty settings. First, we introduce linguistic uncertainty cues through prompt modifications to simulate ambiguous clinical contexts. Second, we construct an answer removal setting, wherein the correct option is deliberately excluded mandating the model to recognize insufficient information and abstain. We analyze both model accuracy and confidence behavior using multiple calibration metrics including calibration gap, Expected Calibration Error (ECE), and Unsafe Confident Error Rate (UCER) across 500 medical questions. Our results reveal a consistent failure mode, i.e., although accuracy degrades under increasing uncertainty, model confidence remains misaligned with accuracy. This leads to a substantial increase in unsafe confident errors, indicating that model confidence remains largely insensitive to clinically meaningful information loss. Furthermore, we observe significant variation across models in their ability to abstain when the correct answer is unavailable, with some models persistently producing high confidence hallucinated answers. These findings expose critical limitations in the epistemic reliability of current LLMs and highlight the need for uncertainty aware evaluation methods prior to their deployment in clinical workflows.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.09080 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:49:41.910Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).