Metraipolis › archive › dc_a73e992f5137

Would an LLM Match into a Residency? Uncertainty, Reasoning and Image Use of Open Language Models on the Spanish MIR Examination — arXiv:2610.31586

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.31586
License (as archived)
CC0-1.0
Archived
2026-10-25T08:35:40.251Z
SHA-256 of the archived text
cc54b5efe659a7ad3fe014b6eddeeff7908c0e3a23dafa827eb0e4fbcc4d15bb
Stable URL of this page
https://metraipolis.com/documento/dc_a73e992f5137

How to cite

Would an LLM Match into a Residency? Uncertainty, Reasoning and Image Use of Open Language Models on the Spanish MIR Examination — arXiv:2610.31586 — arXiv (en), https://arxiv.org/abs/2610.31586 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a73e992f5137 (Metraipolis archive dc_a73e992f5137, sha256 cc54b5efe659a7ad3fe014b6eddeeff7908c0e3a23dafa827eb0e4fbcc4d15bb)
BibTeX
@misc{dca73e992f5137,
  title = {Would an LLM Match into a Residency? Uncertainty, Reasoning and Image Use of Open Language Models on the Spanish MIR Examination — arXiv:2610.31586},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.31586},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a73e992f5137; sha256 cc54b5efe659a7ad3fe014b6eddeeff7908c0e3a23dafa827eb0e4fbcc4d15bb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31586. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Accuracy on medical examinations is common evidence offered for the clinical use of language models, yet a health system that intends to run an open model on its own hardware needs to know considerably more: whether the model recognizes when it is likely to be wrong, whether medical fine-tuning improves this ability, whether its correct answers rest on sound clinical reasoning, or whether it actually uses the images it is given. We examine these questions for four open models of 8 to 27 billion parameters on the Spanish MIR residency examination, scoring 1,552 questions from eight editions under the exam's own rules. Answering every question, the three larger models would rank among roughly the first 600 to 3,000 candidates, with 41 to 43 specialties still open to them. Abstention, however, never pays, because even their least confident answers are right more often than the 25% break-even. Medical fine-tuning reduced overconfidence at the cost of accuracy, without improving how well confidence separates right from wrong answers. Moreover, 13-38% of correct answers rejected a distractor for a clinically unsound reason that confidence did not reflect, and removing or swapping the provided contextual images left accuracy unchanged.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.31586 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-25T08:31:13.273Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).