Metraipolis › archive › dc_6ac8f2cf1d97
Beyond Imitation: A Framework and Benchmark for LLM-Assisted Peer Review — arXiv:2610.11087
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.11087
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T05:47:26.669Z
- SHA-256 of the archived text
de4f11404e6d89e40bfda774b7bfefc023047b0198207b1ca56890075dc88068- Stable URL of this page
https://metraipolis.com/documento/dc_6ac8f2cf1d97
How to cite
Beyond Imitation: A Framework and Benchmark for LLM-Assisted Peer Review — arXiv:2610.11087 — arXiv (en), https://arxiv.org/abs/2610.11087 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6ac8f2cf1d97 (Metraipolis archive dc_6ac8f2cf1d97, sha256 de4f11404e6d89e40bfda774b7bfefc023047b0198207b1ca56890075dc88068)
BibTeX
@misc{dc6ac8f2cf1d97,
title = {Beyond Imitation: A Framework and Benchmark for LLM-Assisted Peer Review — arXiv:2610.11087},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.11087},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6ac8f2cf1d97; sha256 de4f11404e6d89e40bfda774b7bfefc023047b0198207b1ca56890075dc88068},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.11087. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The rapid growth of scientific publishing has strained peer review, particularly in machine learning, raising concerns about declining review quality and increasing reviewer workload. Large language models (LLMs) have been proposed as automated review assistants, yet their evaluation has focused largely on imitating human-written reviews rather than supporting the core functions of peer review. Here, we introduce a verification-centric perspective on LLM-assisted peer review, emphasizing error detection as a critical and resource-intensive task. We present a scalable benchmark that evaluates review systems' ability to identify logical contradictions, constructed through synthetic insertion of errors into conference papers, yielding unambiguous evaluation targets and enabling systematic comparison. We further propose a Multi-Layered Review (MLR) framework that prioritizes detailed manuscript comprehension before review generation, aligning more closely with human reviewing practices while improving token efficiency. Across evaluations, our approach demonstrates strong alignment with human review scores, achieves high error detection performance, and provides complementary perspectives on reviewer focus. These improvements can be attributed to both the choice of the underlying LLM and the design of our system. At the same time, we corroborate persistent vulnerabilities to adversarial manipulation, underscoring the need for robustness in automated review systems. Our findings highlight the importance of rigorous, error-focused evaluation to guide responsible deployment of LLM-based tools in peer review and other critical scientific workflows. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.11087 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T05:47:00.514Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).