Metraipolis › archive › dc_08e1693fbdaf
A Benchmark Framework for Screening Automation in Systematic Reviews — arXiv:2609.30298
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.30298
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T02:03:55.882Z
- SHA-256 of the archived text
79a63d0e1884aa8d7e5c930106aff07fb3d955713353c8e96ff48196833ee070- Stable URL of this page
https://metraipolis.com/documento/dc_08e1693fbdaf
How to cite
A Benchmark Framework for Screening Automation in Systematic Reviews — arXiv:2609.30298 — arXiv (en), https://arxiv.org/abs/2609.30298 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_08e1693fbdaf (Metraipolis archive dc_08e1693fbdaf, sha256 79a63d0e1884aa8d7e5c930106aff07fb3d955713353c8e96ff48196833ee070)
BibTeX
@misc{dc08e1693fbdaf,
title = {A Benchmark Framework for Screening Automation in Systematic Reviews — arXiv:2609.30298},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.30298},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_08e1693fbdaf; sha256 79a63d0e1884aa8d7e5c930106aff07fb3d955713353c8e96ff48196833ee070},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30298. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Systematic reviews (SR) are essential for evidence-based research, but their screening phase is highly time-consuming and labor-intensive. Large language models (LLMs) offer a promising opportunity to reduce this workload by assisting with article relevance classification. However, existing evaluation approaches often rely on traditional metrics that may be misleading for highly imbalanced SR screening datasets. This paper presents a benchmark dataset of $45\,064$ labeled entries for evaluating LLM performance in SR screening across 32 curated secondary studies. It proposes an evaluation framework that accounts for class imbalance, i.e., the natural prevalence of excluded articles relative to included articles in SRs. It also introduces PromptSR, a tool designed to support prompt experimentation, experiment management, and result analysis for LLM-based screening. We also present a use case demonstrating the application of SRBench and PromptSR. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.30298 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T01:38:49.422Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).