Metraipolis › archive › dc_a353e17529ed
A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Review — arXiv:2609.39027
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.39027
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T13:13:39.463Z
- SHA-256 of the archived text
77751e3d1cadfe825f53d958cb563c1bbfe8b77d170a3462dd216d53df5fd93a- Stable URL of this page
https://metraipolis.com/documento/dc_a353e17529ed
How to cite
A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Review — arXiv:2609.39027 — arXiv (en), https://arxiv.org/abs/2609.39027 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a353e17529ed (Metraipolis archive dc_a353e17529ed, sha256 77751e3d1cadfe825f53d958cb563c1bbfe8b77d170a3462dd216d53df5fd93a)
BibTeX
@misc{dca353e17529ed,
title = {A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Review — arXiv:2609.39027},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.39027},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a353e17529ed; sha256 77751e3d1cadfe825f53d958cb563c1bbfe8b77d170a3462dd216d53df5fd93a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39027. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
AI reviewers can assign different judgments to manuscripts that report the same science in different wording, potentially rewarding rhetorical optimization over scientific improvement. We formulate Rhetorical Robustness as the joint requirement of stability across content-preserving rewrites and discrimination across papers. We introduce RobustReview, a controlled full-manuscript benchmark with 1,260 manuscript versions, and evaluate 30 reviewer configurations. The benchmark reveals false robustness, where low rewrite sensitivity coincides with score collapse across papers, and shows that human alignment and rhetorical robustness rank reviewers differently. Moreover, the evaluated content-focused prompting protocol does not consistently improve robustness across backbones. Motivated by these findings, we introduce SciCore, a dual-branch reviewer that averages a full-manuscript judgment with a judgment based on an extracted, structured science core. This design combines manuscript-level assessment with a content-normalized view intended to reduce rhetorical sensitivity. In our primary GPT-5.5 comparison, SciCore achieves a leading joint stability-discrimination profile among the benchmarked reviewers while maintaining competitive human alignment. These results identify rhetorical robustness as a distinct evaluation target and demonstrate the potential of science-core review to improve it. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.39027 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T12:44:51.787Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).