Metraipolis › archive › dc_020250b050d2

Medical Causal Hypothesis Verification with Large Language Models — arXiv:2609.00063

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.00063
License (as archived)
CC0-1.0
Archived
2026-10-17T06:32:40.140Z
SHA-256 of the archived text
3c8a3c01c211df4569c212b42dcca35dec00e09166bbccffd032f09e66af861d
Stable URL of this page
https://metraipolis.com/documento/dc_020250b050d2

How to cite

Medical Causal Hypothesis Verification with Large Language Models — arXiv:2609.00063 — arXiv (en), https://arxiv.org/abs/2609.00063 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_020250b050d2 (Metraipolis archive dc_020250b050d2, sha256 3c8a3c01c211df4569c212b42dcca35dec00e09166bbccffd032f09e66af861d)
BibTeX
@misc{dc020250b050d2,
  title = {Medical Causal Hypothesis Verification with Large Language Models — arXiv:2609.00063},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.00063},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_020250b050d2; sha256 3c8a3c01c211df4569c212b42dcca35dec00e09166bbccffd032f09e66af861d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.00063. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The growing use of large language models (LLMs) for search and information retrieval underscores the need to evaluate their reliability in high-stakes domains such as healthcare. Although LLMs can effectively answer questions about diseases, symptoms, and treatments, their ability to accurately assess causal relationships and ground their conclusions in verified scientific evidence remains unclear. Here, we present a preliminary, small-scale study that investigates the accuracy of LLMs in evaluating causal medical claims and supporting them with peer-reviewed research. We propose an evaluation framework for causal hypothesis verification that can be used to systematically track the performance of existing and future LLMs. We assess the performance of eight LLMs on 17 medical causal hypotheses to evaluate whether they can reliably verify these hypotheses using scientific evidence from the literature. We systematically annotate the scientific evidence they provide according to six criteria (a total of 1,067 annotation points) and assess them with nine evaluation metrics. Our analysis shows that while LLMs exhibit strong recall, they often perform poorly at providing valid scientific articles and evidence for support and at rejecting unsupported hypotheses. These findings highlight a critical limitation of current LLMs, as they cannot yet be trusted fully to verify causal relationships from the biomedical literature. This work underscores the need for rigorous evaluation before using LLMs for search and retrieval in healthcare settings.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.00063 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T06:15:12.727Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).