Metraipolis › archive › dc_172d84bcf17f

Evaluating Retrieval Quality in Retrieval-Augmented Generation (2024) · OpenAlex W4400526126

Source
OpenAlex
Original
https://doi.org/10.1145/3626772.3657957
License (as archived)
CC-BY-4.0
Archived
2026-10-07T12:13:54.420Z
SHA-256 of the archived text
057e11f944730ec4cf3992defbba0bbc791a2f5665c855c749bbb8404dab01ad
Stable URL of this page
https://metraipolis.com/documento/dc_172d84bcf17f

How to cite

Evaluating Retrieval Quality in Retrieval-Augmented Generation (2024) · OpenAlex W4400526126 — OpenAlex, https://doi.org/10.1145/3626772.3657957 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_172d84bcf17f (Metraipolis archive dc_172d84bcf17f, sha256 057e11f944730ec4cf3992defbba0bbc791a2f5665c855c749bbb8404dab01ad)
BibTeX
@misc{dc172d84bcf17f,
  title = {Evaluating Retrieval Quality in Retrieval-Augmented Generation (2024) · OpenAlex W4400526126},
  howpublished = {OpenAlex},
  url = {https://doi.org/10.1145/3626772.3657957},
  note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_172d84bcf17f; sha256 057e11f944730ec4cf3992defbba0bbc791a2f5665c855c749bbb8404dab01ad},
  year = {2026}
}

Archived text

Reproduced under CC-BY-4.0. Attribution: OpenAlex, https://doi.org/10.1145/3626772.3657957. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Opera scientifica in accesso aperto: scheda dai metadati OpenAlex (conference-paper).
Titolo: Evaluating Retrieval Quality in Retrieval-Augmented Generation
Autori: Alireza Salemi, Hamed Zamani
Anno: 2024 · Sede: International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · Lingua: en
DOI: https://doi.org/10.1145/3626772.3657957
Accesso aperto: hybrid · testo: https://doi.org/10.1145/3626772.3657957 · licenza del testo: cc-by
Citazioni ricevute (OpenAlex): 151
Temi (OpenAlex): Information Retrieval and Search Behavior; Topic Modeling; Multimodal Machine Learning Applications

Abstract (testo degli autori, licenza cc-by):
Evaluating retrieval-augmented generation (RAG) presents challenges, particularly for retrieval models within these systems. Traditional end-to-end evaluation methods are computationally expensive. Furthermore, evaluation of the retrieval model's performance based on query-document relevance labels shows a small correlation with the RAG system's downstream performance. We propose a novel evaluation approach, eRAG, where each document in the retrieval list is individually utilized by the large language model within the RAG system. The output generated for each document is then evaluated based on the downstream task ground truth labels. In this manner, the downstream performance for each document serves as its relevance label. We employ various downstream task metrics to obtain document-level annotations and aggregate them using set-based or ranking metrics. Extensive experiments on a wide range of datasets demonstrate that eRAG achieves a higher correlation with downstream RAG performance compared to baseline methods, with improvements in Kendall's tau correlation ranging from 0.168 to 0.494. Additionally, eRAG offers significant computational advantages, improving runtime and consuming up to 50 times less GPU memory than end-to-end evaluation.

Fonte: https://openalex.org/W4400526126 — metadati OpenAlex (CC0 1.0), consultati il 2026-10-07 via api.openalex.org. Interrogazione: «retrieval augmented generation».