Metraipolis › archive › dc_2f1ae8ac8aae

Automating the triage of rheumatology outpatient referrals: a comparative evaluation of 23 large language models under simple and advanced prompting (2026) · DOI 10.21203/rs.3.rs-10563856/v1

Source
Metraipolis archive
Original
https://doi.org/10.21203/rs.3.rs-10563856/v1
License (as archived)
CC-BY-4.0
Archived
2026-10-08T07:05:50.463Z
SHA-256 of the archived text
72f2231624ea65f28c1a42eae447bd91174c0cd7394bcd5db23a4878455a31c0
Stable URL of this page
https://metraipolis.com/documento/dc_2f1ae8ac8aae

How to cite

Automating the triage of rheumatology outpatient referrals: a comparative evaluation of 23 large language models under simple and advanced prompting (2026) · DOI 10.21203/rs.3.rs-10563856/v1 — Metraipolis archive, https://doi.org/10.21203/rs.3.rs-10563856/v1 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_2f1ae8ac8aae (Metraipolis archive dc_2f1ae8ac8aae, sha256 72f2231624ea65f28c1a42eae447bd91174c0cd7394bcd5db23a4878455a31c0)
BibTeX
@misc{dc2f1ae8ac8aae,
  title = {Automating the triage of rheumatology outpatient referrals: a comparative evaluation of 23 large language models under simple and advanced prompting (2026) · DOI 10.21203/rs.3.rs-10563856/v1},
  howpublished = {Metraipolis archive},
  url = {https://doi.org/10.21203/rs.3.rs-10563856/v1},
  note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_2f1ae8ac8aae; sha256 72f2231624ea65f28c1a42eae447bd91174c0cd7394bcd5db23a4878455a31c0},
  year = {2026}
}

Archived text

Reproduced under CC-BY-4.0. Attribution: Metraipolis archive, https://doi.org/10.21203/rs.3.rs-10563856/v1. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Articolo recente con licenza aperta: scheda dai metadati Crossref (posted-content).
Titolo: Automating the triage of rheumatology outpatient referrals: a comparative evaluation of 23 large language models under simple and advanced prompting
Autori: Lynden Roberts
Pubblicato: 2026-08-25 · editore: Springer Science and Business Media LLC
DOI: https://doi.org/10.21203/rs.3.rs-10563856/v1
Licenza dell'articolo: CC-BY-4.0

Abstract (testo degli autori, CC-BY-4.0):
Abstract Objective. Triage of rheumatology outpatient referrals is a high-volume administrative task that consumes senior specialist time without advancing patient care. The human triage system is only moderately accurate and reproducible. We assessed whether contemporary large language models (LLMs) are able to perform well enough to support automating this task in practice. In addition, the effects of different prompting techniques on triage accuracy and cost was assessed to help identify to optimal approach. Methods. Twenty referral scenarios spanning the urgency spectrum, based on real referrals were created by a certified Australian rheumatologist. Four rheumatologists triaged all cases independently and blinded, to produce a consensus reference standard. Twenty-three LLMs each triaged every referral into one of five urgency categories, three times (1380 outputs per condition). The experiment was run with a simple prompt and repeated with a advanced prompt supplying explicit triage expectations and worked examples. Results. All 2760 attempts returned valid categories. Under the simple prompt, performance separated into distinct tiers, larger models were more accurate (Spearman rho = 0.42; P=.047) and accuracy tracked cost. Advanced prompting minimised between-model variance in accuracy 5.3-fold (0.014 to 0.003; Levene P=.01), abolished the size-accuracy association (rho=-0.05; P=.83) and removed the accuracy-cost relationship. Leading models matched expert consensus on most cases, within or above the range reported for human triage. Under-triage errors persisted with some LLMs. Conclusion. Contemporary LLMs categorise rheumatology referral urgency as well or better than published human triage systems. Advanced LLM prompting methods substitute for the reasoning capability of larger models, suggesting that LLM performance on this task may not require the most expensive models. The tools to automate this administrative task appear to already exist. Strong candidate LLMs that might serve a production ready solution have been identified.

Fonte: https://api.crossref.org/works/10.21203%2Frs.3.rs-10563856%2Fv1 — metadati Crossref, consultati il 2026-10-08. Interrogazione: «evaluation of large language models».