Metraipolis › archive › dc_5110eb204dae

Benchmarking Retrieval-Augmented Generation for Medicine (2024) · OpenAlex W4402670290

Source
OpenAlex
Original
https://doi.org/10.18653/v1/2024.findings-acl.372
License (as archived)
CC-BY-4.0
Archived
2026-10-07T11:17:48.548Z
SHA-256 of the archived text
c2dc474c81eb93e1b1ef107265d892fbe18efbb156e6e5acde81933e4167c168
Stable URL of this page
https://metraipolis.com/documento/dc_5110eb204dae

How to cite

Benchmarking Retrieval-Augmented Generation for Medicine (2024) · OpenAlex W4402670290 — OpenAlex, https://doi.org/10.18653/v1/2024.findings-acl.372 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_5110eb204dae (Metraipolis archive dc_5110eb204dae, sha256 c2dc474c81eb93e1b1ef107265d892fbe18efbb156e6e5acde81933e4167c168)
BibTeX
@misc{dc5110eb204dae,
  title = {Benchmarking Retrieval-Augmented Generation for Medicine (2024) · OpenAlex W4402670290},
  howpublished = {OpenAlex},
  url = {https://doi.org/10.18653/v1/2024.findings-acl.372},
  note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_5110eb204dae; sha256 c2dc474c81eb93e1b1ef107265d892fbe18efbb156e6e5acde81933e4167c168},
  year = {2026}
}

Archived text

Reproduced under CC-BY-4.0. Attribution: OpenAlex, https://doi.org/10.18653/v1/2024.findings-acl.372. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Opera scientifica in accesso aperto: scheda dai metadati OpenAlex (conference-paper).
Titolo: Benchmarking Retrieval-Augmented Generation for Medicine
Autori: Guangzhi Xiong, Qiao Jin, Zhiyong Lu, Aidong Zhang
Anno: 2024 · Lingua: en
DOI: https://doi.org/10.18653/v1/2024.findings-acl.372
Accesso aperto: gold · testo: https://doi.org/10.18653/v1/2024.findings-acl.372 · licenza del testo: cc-by
Citazioni ricevute (OpenAlex): 273
Temi (OpenAlex): Topic Modeling; Information Retrieval and Search Behavior; Artificial Intelligence in Healthcare and Education

Abstract (testo degli autori, licenza cc-by):
While large language models (LLMs) have achieved state-of-the-art performance on a wide range of medical question answering (QA) tasks, they still face challenges with hallucinations and outdated knowledge.Retrievalaugmented generation (RAG) is a promising solution and has been widely adopted.However, a RAG system can involve multiple flexible components, and there is a lack of best practices regarding the optimal RAG setting for various medical purposes.To systematically evaluate such systems, we propose the Medical Information Retrieval-Augmented Generation Evaluation (MIRAGE), a first-of-its-kind benchmark including 7,663 questions from five medical QA datasets.Using MIRAGE, we conducted large-scale experiments with over 1.8 trillion prompt tokens on 41 combinations of different corpora, retrievers, and backbone LLMs through the MEDRAG toolkit introduced in this work.Overall, MEDRAG improves the accuracy of six different LLMs by up to 18% over chain-of-thought prompting, elevating the performance of GPT-3.5 and Mixtral to GPT-4level.Our results show that the combination of various medical corpora and retrievers achieves the best performance.In addition, we discovered a log-linear scaling property and the "lostin-the-middle" effects in medical RAG.We believe our comprehensive evaluations can serve as practical guidelines for implementing RAG systems for medicine 1 .

Fonte: https://openalex.org/W4402670290 — metadati OpenAlex (CC0 1.0), consultati il 2026-10-07 via api.openalex.org. Interrogazione: «retrieval augmented generation».