Metraipolis › archive › dc_2bf09f1371f1

A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined — arXiv:2610.01938

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.01938
License (as archived)
CC0-1.0
Archived
2026-10-06T12:15:37.177Z
SHA-256 of the archived text
2c65712d1b35592624a71f3ce0d988eac8104e0d47e4fd3046dbab299f671fab
Stable URL of this page
https://metraipolis.com/documento/dc_2bf09f1371f1

How to cite

A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined — arXiv:2610.01938 — arXiv (en), https://arxiv.org/abs/2610.01938 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2bf09f1371f1 (Metraipolis archive dc_2bf09f1371f1, sha256 2c65712d1b35592624a71f3ce0d988eac8104e0d47e4fd3046dbab299f671fab)
BibTeX
@misc{dc2bf09f1371f1,
  title = {A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined — arXiv:2610.01938},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.01938},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2bf09f1371f1; sha256 2c65712d1b35592624a71f3ce0d988eac8104e0d47e4fd3046dbab299f671fab},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01938. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Exam-style accuracy does not establish whether large language models (LLMs) reason well over clinical records. We define clinical reasoning as integrating and updating evidence across time and sources to form, revise and justify a patient's problem representation and a defensible plan. This structured narrative review maps three literatures: medical education assessment instruments, clinical LLM benchmarks published from 2023 onwards, and general-domain methods for evaluating long-form generation. We examine six dimensions: problem representation, temporal synthesis, differential and management reasoning, counterfactual reasoning, calibrated uncertainty, and reasoning faithfulness. Preprints are included and flagged. No single instrument covers all six dimensions. Problem representation and differential or management reasoning are reasonably covered, although reliability varies by instrument and setting. TIMER-Eval targets temporal synthesis, and ER-Reason assesses sequential diagnostic belief updating. Dedicated uncertainty and counterfactual evaluations are emerging, but their applicability to longitudinal free-text reasoning remains limited. Factual completeness is well theorised in general-domain evaluation, with early clinical evidence of important omissions. Faithfulness remains the weakest dimension, with one identified clinical causal-ablation study on multiple-choice questions. Existing tools should be combined through binary rubric items, separate completeness and correctness scores, case-specific importance weighting with non-compensable safety caps, temporal order-consistency checks, and chance-corrected reliability reporting. Further design work is needed for calibrated uncertainty, counterfactual reasoning and faithfulness over longitudinal free-text records. This review provides a design rationale, not a validated instrument.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.01938 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:28.230Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).