Metraipolis › archive › dc_e332ce734559
Breaking the Illusion of Review Reliability under Static Evaluation: SCOPE Fuzzing for LLM-based Scientific Reviewers — arXiv:2609.37097
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.37097
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:23:28.796Z
- SHA-256 of the archived text
86ccb0ccb398dfc9cbd3df9cf7d0a6f5058260b863ddb4b8397c803ccc0019b2- Stable URL of this page
https://metraipolis.com/documento/dc_e332ce734559
How to cite
Breaking the Illusion of Review Reliability under Static Evaluation: SCOPE Fuzzing for LLM-based Scientific Reviewers — arXiv:2609.37097 — arXiv (en), https://arxiv.org/abs/2609.37097 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e332ce734559 (Metraipolis archive dc_e332ce734559, sha256 86ccb0ccb398dfc9cbd3df9cf7d0a6f5058260b863ddb4b8397c803ccc0019b2)
BibTeX
@misc{dce332ce734559,
title = {Breaking the Illusion of Review Reliability under Static Evaluation: SCOPE Fuzzing for LLM-based Scientific Reviewers — arXiv:2609.37097},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.37097},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e332ce734559; sha256 86ccb0ccb398dfc9cbd3df9cf7d0a6f5058260b863ddb4b8397c803ccc0019b2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37097. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The rapid growth of submissions and reviewing workload has accelerated the use of large language models (LLMs) in peer review. Prior studies suggest that LLM-based reviewers can penalize content perturbations, such as overclaiming, indicating a certain degree of reliability. Yet these conclusions are largely based on a narrow set of perturbation strategies instantiated with static templates, providing limited evidence of actual reliability. In this paper, we construct a three-level evaluation framework covering perturbations to surface presentation, argumentative logic, and value judgment. Experiments on representative LLM-based reviewers reveal two limitations of static evaluation: stratified vulnerability, where perturbation effects depend on whether the paper's original review score is high or low, and perturbation undercoverage, where a single template misses vulnerabilities exposed by diverse realizations. To address these limitations, we propose SCOPE-Fuzzer, a strategy-aware fuzzer that combines feedback-driven strategy selection with adaptive mutation of paper content. By iteratively probing reviewers with dynamic perturbations, SCOPE-Fuzzer consistently uncovers vulnerabilities overlooked by static evaluation and other baselines. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.37097 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T15:54:53.587Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).