Metraipolis › archive › dc_3ff234767da0

Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts — arXiv:2609.38600

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.38600
License (as archived)
CC0-1.0
Archived
2026-10-07T12:41:57.803Z
SHA-256 of the archived text
11f306614af853c1d952d839eabff286dd56e553b7f6c417e870704921c78af9
Stable URL of this page
https://metraipolis.com/documento/dc_3ff234767da0

How to cite

Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts — arXiv:2609.38600 — arXiv (en), https://arxiv.org/abs/2609.38600 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3ff234767da0 (Metraipolis archive dc_3ff234767da0, sha256 11f306614af853c1d952d839eabff286dd56e553b7f6c417e870704921c78af9)
BibTeX
@misc{dc3ff234767da0,
  title = {Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts — arXiv:2609.38600},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.38600},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3ff234767da0; sha256 11f306614af853c1d952d839eabff286dd56e553b7f6c417e870704921c78af9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38600. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As large language models (LLMs) are increasingly used in clinical settings, it is critical to evaluate their reliability under realistic variation in clinical text. We study this question in clinical triage, comparing LLMs to practicing physicians under text perturbations that preserve the underlying clinical setting. We introduce a benchmark of over 6,000 clinical scenarios, 7,000 physician annotations, and 225,000 model responses. Using this benchmark, we make two key observations. First, LLMs are more likely than physicians to recommend unnecessary care at baseline, and this tendency increases under perturbed inputs. Further, we find that LLM recommendations are more sensitive to gender and tone perturbations than human recommendations. Together, these results demonstrate that LLMs can vary under clinically irrelevant textual changes, highlighting the need for deployment-oriented evaluations grounded in expert physician behavior.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.38600 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:13.648Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).