Metraipolis › archive › dc_bd5255c04ce0
Biomedical Reference Generation Remains Unreliable across 26 Large Language Models — arXiv:2609.14988
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.14988
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T23:15:40.384Z
- SHA-256 of the archived text
1eccd994e146b7638c7b26f7f4136dfb25fb67deb47e40cc91a8d295e4e7974a- Stable URL of this page
https://metraipolis.com/documento/dc_bd5255c04ce0
How to cite
Biomedical Reference Generation Remains Unreliable across 26 Large Language Models — arXiv:2609.14988 — arXiv (en), https://arxiv.org/abs/2609.14988 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bd5255c04ce0 (Metraipolis archive dc_bd5255c04ce0, sha256 1eccd994e146b7638c7b26f7f4136dfb25fb67deb47e40cc91a8d295e4e7974a)
BibTeX
@misc{dcbd5255c04ce0,
title = {Biomedical Reference Generation Remains Unreliable across 26 Large Language Models — arXiv:2609.14988},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.14988},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bd5255c04ce0; sha256 1eccd994e146b7638c7b26f7f4136dfb25fb67deb47e40cc91a8d295e4e7974a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.14988. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Background. Large language models are increasingly used to help write biomedical text but may fabricate references to nonexistent work. How often large language models do so is not well characterized. Methods. We prompted 26 language models from eight developers (2023 to 2026) to supply a missing reference for each of 69 biomedical passages across ten domains. References were classified as verifiable (real paper with a resolving identifier), partial matches (real paper without a resolving identifier), fabricated (no matching indexed paper), or declined (the model refused to supply a reference). A reference was considered correct in every evaluated bibliographic field only when it was verifiable and its journal, year, and listed authors matched those of the cited paper. Results. Fabrication ranged from 10.2% (Claude Opus 4.8, which declined 52.1% of prompts) to 98.4% (Ministral 3B, which produced no verifiable reference). Claude Opus 4.6 and Claude Sonnet 4.5 produced similar proportions of verifiable references (77.6% and 76.6%) but named authors correctly in 78.7% and 28.7% of author-evaluable verifiable references, respectively, and were correct in every evaluated field in 54.6% and 19.9% of responses. GPT-5.5 was correct in every field in 48.1%. Across all models, 55.4% of responses were fabricated and 14.9% were correct in every field. Among the five tested models first released in 2026, the corresponding proportions were 35.3% and 31.8%, respectively. Conclusions. Fabrication remained common, and no model was correct in every evaluated bibliographic field in more than 54.6% of responses. Models that identify real papers may still misstate their metadata, so references produced with model assistance require verification before use. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.14988 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:51:09.919Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).