Metraipolis › archive › dc_5be0b58d01eb
Use and Effects of LLMs in Peer Review: A Randomized Experiment and Survey at ICML 2026 — arXiv:2609.19420
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.19420
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T04:00:26.908Z
- SHA-256 of the archived text
9891dc1c2b1e070b3afd5e8ca0b96dcd0abf24f5c12bbb3a3c6e99b250165392- Stable URL of this page
https://metraipolis.com/documento/dc_5be0b58d01eb
How to cite
Use and Effects of LLMs in Peer Review: A Randomized Experiment and Survey at ICML 2026 — arXiv:2609.19420 — arXiv (en), https://arxiv.org/abs/2609.19420 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5be0b58d01eb (Metraipolis archive dc_5be0b58d01eb, sha256 9891dc1c2b1e070b3afd5e8ca0b96dcd0abf24f5c12bbb3a3c6e99b250165392)
BibTeX
@misc{dc5be0b58d01eb,
title = {Use and Effects of LLMs in Peer Review: A Randomized Experiment and Survey at ICML 2026 — arXiv:2609.19420},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.19420},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5be0b58d01eb; sha256 9891dc1c2b1e070b3afd5e8ca0b96dcd0abf24f5c12bbb3a3c6e99b250165392},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19420. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LLMs are rapidly reshaping peer review, making it important to understand how reviewers use them in practice and how different LLM-use policies affect review outcomes. We investigate these questions through a randomized experiment and an anonymous post-survey at ICML 2026, a major machine learning conference involving over 24,000 papers and 17,000 reviewers. Reviewers were assigned to either a conservative policy prohibiting all LLM use or a permissive policy allowing limited assistance, with randomization among a subset of main-track papers and reviewers. Policy assignment had near-zero effects on final paper decisions, paper scores, and reviewer confidence, although reviews under the permissive policy were 5.5-7% longer. Post-survey responses (N=1,486) revealed diverse attitudes toward LLMs and substantial noncompliance: 22.5% of conservative-policy reviewers reported using an LLM despite the prohibition, and 36.5% of permissive-policy reviewers reported at least one explicitly disallowed use. We discuss implications for future peer-review policy and tool design. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.HC). Categorie arXiv: Human-Computer Interaction (cs.HC). Pagina del lavoro: https://arxiv.org/abs/2609.19420 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T03:28:23.621Z dalla pagina https://arxiv.org/catchup/cs.HC/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).