Metraipolis › archive › dc_ad66a23f37cf
Beyond Human-Likeness: Mapping the Scientific Critique Profiles of LLMs and Human Reviewers — arXiv:2609.01895
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.01895
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T04:33:40.522Z
- SHA-256 of the archived text
3bcc0b8a8310ad8a707c15a0b56e98df6f318c7731d29f283a92a2364c28fb09- Stable URL of this page
https://metraipolis.com/documento/dc_ad66a23f37cf
How to cite
Beyond Human-Likeness: Mapping the Scientific Critique Profiles of LLMs and Human Reviewers — arXiv:2609.01895 — arXiv (en), https://arxiv.org/abs/2609.01895 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ad66a23f37cf (Metraipolis archive dc_ad66a23f37cf, sha256 3bcc0b8a8310ad8a707c15a0b56e98df6f318c7731d29f283a92a2364c28fb09)
BibTeX
@misc{dcad66a23f37cf,
title = {Beyond Human-Likeness: Mapping the Scientific Critique Profiles of LLMs and Human Reviewers — arXiv:2609.01895},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.01895},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ad66a23f37cf; sha256 3bcc0b8a8310ad8a707c15a0b56e98df6f318c7731d29f283a92a2364c28fb09},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.01895. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) are increasingly discussed as tools for peer review, but their value is often assessed through human-likeness, perceived usefulness, or textual overlap with reviewer comments. This study shifts attention from whether LLMs resemble human reviewers to what functions of scientific critique they perform. Using ICLR 2025 peer-review data, we compare human reviews with LLM reviews generated under baseline and expert prompts. We operationalize scientific critique through two review acts, weakness critique and scientific questioning, and annotate point-level review text using five theory-guided frameworks: Anderson's knowledge types, Toulmin's argumentation model, Graesser's question depth, SOLO cognitive complexity, and Hattie's feedback functions. The results reveal a differentiated critique profile. Human reviews placed greater emphasis on scientific framing and revision guidance, more often identifying higher-order weaknesses and asking questions oriented toward improvement. LLM reviews showed higher rates of explanatory depth, integrative reasoning, and explicit argument structuring. Expert prompting did not make LLM critique uniformly more human-like; it partially narrowed some gaps but mainly amplified LLM-specific tendencies toward integration and formal argumentation. These findings show that LLM-assisted peer review changes the functional composition of review text, making it important to distinguish LLM-amplified critique from areas requiring human prioritization and accountable judgement. Tema degli esploratori: ricerca e scienza aperta (retro-catalogo arXiv: annunci del 2026-09-03 in cs.DL). Categorie arXiv: Digital Libraries (cs.DL). Pagina del lavoro: https://arxiv.org/abs/2609.01895 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T04:08:36.699Z dalla pagina https://arxiv.org/catchup/cs.DL/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).