Metraipolis › archive › dc_2310be71bef7
Errors of LLM-Assisted Literature Retrieval in Environmental Science: A Comparison Study of Abstract versus Full-text Based Prompts — arXiv:2610.05690
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.05690
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:29:38.133Z
- SHA-256 of the archived text
52c0460c9bde6dea97138ae6a36206f11a32a7fdc574075a6333ba4f04131816- Stable URL of this page
https://metraipolis.com/documento/dc_2310be71bef7
How to cite
Errors of LLM-Assisted Literature Retrieval in Environmental Science: A Comparison Study of Abstract versus Full-text Based Prompts — arXiv:2610.05690 — arXiv, https://arxiv.org/abs/2610.05690 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2310be71bef7 (Metraipolis archive dc_2310be71bef7, sha256 52c0460c9bde6dea97138ae6a36206f11a32a7fdc574075a6333ba4f04131816)
BibTeX
@misc{dc2310be71bef7,
title = {Errors of LLM-Assisted Literature Retrieval in Environmental Science: A Comparison Study of Abstract versus Full-text Based Prompts — arXiv:2610.05690},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.05690},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2310be71bef7; sha256 52c0460c9bde6dea97138ae6a36206f11a32a7fdc574075a6333ba4f04131816},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05690. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) are increasingly used for literature search and synthesis. However, it is unclear whether they retrieve accurate bibliographic information in environmental science. Therefore, we quantitatively compared the errors of widely used LLM platforms in retrieving references related to original articles from five leading environmental science journals (Energy and Environmental Science, Nature Sustainability, Nature Climate Change, Lancet Planetary Health, and Environmental Science and Technology) published in 2024 to 2025. Claude, ChatGPT, Grok, DeepSeek, Perplexity, and Gemini were used as the LLM platforms. LLMs retrieved 10 references for each of the 50 randomly selected original article using either the article's abstract or its full-text as prompt. The retrieved references were subject to a multimetric score ratio combining validity of bibliographic data, Google Scholar link, digital object identifier, Scopus Electronic Identifier and relevance score (cited by or being the index paper), and the proportion of complete fabrication that failed all metrics. Abstract-only prompt yielded significantly higher accuracy than full-text one. This advantage was confirmed in multilevel mixed-effect multivariable regression after adjusting for journal, platform, and output order. Source journal and the position of a reference within the output list were also independently associated with retrieval accuracy, with lower-listed references associated with lower accuracy. These findings suggest that LLM assisted literature retrieval in environmental science remains moderately accurate and overall inconsistent, varying significantly by platform, journal, prompt type, and output position. Abstract-based prompting, as task-aligned information compression, may outperform full-text one in literature retrieval. Caution should be used when generalizing our findings. Tema degli esploratori: ricerca e scienza aperta (elenco arXiv cs.DL, nuovi lavori del giorno). Categorie arXiv: Digital Libraries (cs.DL) ; Information Retrieval (cs.IR); Machine Learning (cs.LG); Applications (stat.AP); Methodology (stat.ME). Pagina del lavoro: https://arxiv.org/abs/2610.05690 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:26:05.292Z dalla pagina https://arxiv.org/list/cs.DL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).