Metraipolis › archive › dc_ffe20445848e

Peerify: Benchmarking Peer-Review Claim Verification — arXiv:2609.25046

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.25046
License (as archived)
CC0-1.0
Archived
2026-10-08T14:34:19.663Z
SHA-256 of the archived text
03d7c1c3477f0e43a7384123dfe2fa8214f66ba271156dfbf72217ec7719cca0
Stable URL of this page
https://metraipolis.com/documento/dc_ffe20445848e

How to cite

Peerify: Benchmarking Peer-Review Claim Verification — arXiv:2609.25046 — arXiv (en), https://arxiv.org/abs/2609.25046 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ffe20445848e (Metraipolis archive dc_ffe20445848e, sha256 03d7c1c3477f0e43a7384123dfe2fa8214f66ba271156dfbf72217ec7719cca0)
BibTeX
@misc{dcffe20445848e,
  title = {Peerify: Benchmarking Peer-Review Claim Verification — arXiv:2609.25046},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.25046},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ffe20445848e; sha256 03d7c1c3477f0e43a7384123dfe2fa8214f66ba271156dfbf72217ec7719cca0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25046. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Peer review plays a central role in scholarly publishing, yet verifying whether reviewer claims are supported by manuscript evidence remains a largely manual and time-consuming process. We present Peerify, a pipeline for manuscript-grounded verification of peer-review claims. Given a manuscript and a review comment, the Peerify pipeline decomposes reviews into atomic claims, retrieves relevant manuscript evidence, and determines whether each claim is supported by the paper. To support the development and evaluation of the pipeline, we construct a benchmark of 800 claims derived from authentic peer-review interactions collected from NeurIPS 2024 and ICLR 2024, including a 300-claim hand-labeled subset used to audit the automated supervision. We evaluate state-of-the-art language models and retrieval strategies within the Peerify pipeline, together with entailment baselines. Our results demonstrate the importance of retrieval-centered verification and claim decomposition, while highlighting the challenges posed by ambiguous and interpretive reviewer claims. Automated labels agree with human consensus on 90.3% of audited claims ($\kappa = 0.87$), while off-the-shelf entailment models stay below 0.24 macro-F1.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Digital Libraries (cs.DL).
Pagina del lavoro: https://arxiv.org/abs/2609.25046 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T13:46:15.650Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).