Metraipolis › archive › dc_3f291ccd8752
AI-Driven Reproducibility and Analytical Robustness in Scientific Discovery — arXiv:2610.23845
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.23845
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:42:44.713Z
- SHA-256 of the archived text
83b792aceb81ef20d7b02febe33c9dcb08d82ac1ed6a4b1fe1c34442220b74cb- Stable URL of this page
https://metraipolis.com/documento/dc_3f291ccd8752
How to cite
AI-Driven Reproducibility and Analytical Robustness in Scientific Discovery — arXiv:2610.23845 — arXiv (en), https://arxiv.org/abs/2610.23845 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3f291ccd8752 (Metraipolis archive dc_3f291ccd8752, sha256 83b792aceb81ef20d7b02febe33c9dcb08d82ac1ed6a4b1fe1c34442220b74cb)
BibTeX
@misc{dc3f291ccd8752,
title = {AI-Driven Reproducibility and Analytical Robustness in Scientific Discovery — arXiv:2610.23845},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.23845},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3f291ccd8752; sha256 83b792aceb81ef20d7b02febe33c9dcb08d82ac1ed6a4b1fe1c34442220b74cb},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.23845. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reproducible and analytically robust research findings are essential to scientific progress, but evaluating these properties typically requires costly manual effort. We investigate whether LLM agents can help automate reproduction and robustness analysis, using a medical case study grounded in the Surveillance, Epidemiology, and End Results (SEER) database, a widely used U.S. cancer registry underlying tens of thousands of published studies. First, we introduce SEER-Repro, a benchmark of 20 published SEER-based papers together with the data and resources required to reproduce their findings. Second, we create an AI agent that successfully reproduces 77.4% of 2,190 atomic claims, while providing executable baselines for further analysis. Third, we develop an agentic analysis pipeline to test the robustness of the medical findings by identifying analytical choices underlying each finding, proposing defensible alternatives, and rerunning the analysis to determine whether those choices affect the conclusion. Across published papers in SEER-Repro, our pipeline identifies potentially consequential analytical choices in 9.5% of fully reproduced findings and flags them for expert review. These results show how LLM agents can support scientific review by reproducing findings, systematically testing their robustness, and directing expert attention toward potentially consequential analytical choices. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.23845 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:39:10.098Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).