Metraipolis › archive › dc_78112a3544d7
Large Language Model Turnover Undermines Screening for Artificial Intelligence-Assisted Scientific Writing — arXiv:2610.11599
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.11599
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:00:17.121Z
- SHA-256 of the archived text
568f116f81344830e5f630f763718fbb20dd9159992a118639ab26b625baefbe- Stable URL of this page
https://metraipolis.com/documento/dc_78112a3544d7
How to cite
Large Language Model Turnover Undermines Screening for Artificial Intelligence-Assisted Scientific Writing — arXiv:2610.11599 — arXiv (en), https://arxiv.org/abs/2610.11599 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_78112a3544d7 (Metraipolis archive dc_78112a3544d7, sha256 568f116f81344830e5f630f763718fbb20dd9159992a118639ab26b625baefbe)
BibTeX
@misc{dc78112a3544d7,
title = {Large Language Model Turnover Undermines Screening for Artificial Intelligence-Assisted Scientific Writing — arXiv:2610.11599},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.11599},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_78112a3544d7; sha256 568f116f81344830e5f630f763718fbb20dd9159992a118639ab26b625baefbe},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.11599. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Journals and conferences have begun to screen submitted manuscripts for text written using large language models (LLMs). The reliability of this screening rests on benchmark evaluations against a fixed set of LLM versions, while the versions in actual use keep changing. Here we quantify how this LLM turnover affects the screening of scientific manuscripts. We paired 4,000 pre-ChatGPT abstracts from the Proceedings of the National Academy of Sciences with their rewrites by 23 LLM versions from three vendors, released between June 2023 and August 2026. We then trained detectors under maintenance scenarios ranging from a detector retrained on every new version to one trained once and never updated. Detectors trained only on a vendor's past versions can collapse at the boundaries between model generations: calibrated to falsely flag 1% of human-written abstracts, they catch above 99% of rewrites just before the sharpest boundary and 3.8% just after it. Detectors trained on later versions can also miss rewrites of earlier ones. Vocabulary differences between versions largely track where detection transfers and where it fails. In the two screening scenarios we simulated, screens covering all 23 versions either flagged one in eight human-written abstracts or missed one in three rewrites of the newest version. Indeed, a commercial detector missed most rewrites of the version just after the sharpest boundary while flagging almost no human-written abstracts. Research-integrity policy should therefore treat the benchmark accuracy of a detector as provisional, to be re-verified with every LLM release, including earlier versions. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Digital Libraries (cs.DL); Social and Information Networks (cs.SI). Pagina del lavoro: https://arxiv.org/abs/2610.11599 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T05:47:01.794Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).