Metraipolis › archive › dc_023d2f44fb20
Beyond the Text: Verifying That Agent-Written Papers Are Backed by Their Artifacts — arXiv:2609.22111
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22111
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:48:50.444Z
- SHA-256 of the archived text
e2580d1e24a583cb2ddde1fac7f60ca239dd6c88744d1aa9ab47548f71d82830- Stable URL of this page
https://metraipolis.com/documento/dc_023d2f44fb20
How to cite
Beyond the Text: Verifying That Agent-Written Papers Are Backed by Their Artifacts — arXiv:2609.22111 — arXiv (en), https://arxiv.org/abs/2609.22111 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_023d2f44fb20 (Metraipolis archive dc_023d2f44fb20, sha256 e2580d1e24a583cb2ddde1fac7f60ca239dd6c88744d1aa9ab47548f71d82830)
BibTeX
@misc{dc023d2f44fb20,
title = {Beyond the Text: Verifying That Agent-Written Papers Are Backed by Their Artifacts — arXiv:2609.22111},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22111},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_023d2f44fb20; sha256 e2580d1e24a583cb2ddde1fac7f60ca239dd6c88744d1aa9ab47548f71d82830},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22111. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language model agents are increasingly capable of conducting research autonomously, producing research documents alongside the code and experiments that ostensibly support them. Yet whether the reported findings are consistently supported by corresponding implementations and execution evidence remains largely unexplored: existing review practices primarily assess textual quality and cannot reliably identify inconsistencies such as hard-coded metrics, unimplemented methods, or unsupported experimental results. We present ReAgent, an automated auditing framework for assessing the consistency between agent-generated research documents and their associated repositories. ReAgent constructs structured representations of scientific claims from research documents and uses them to guide repository analysis and evidence collection. Static auditing examines whether claimed methodologies, implementations, and experimental configurations are consistently reflected in the repository, while dynamic auditing executes relevant experiments and collects execution evidence to assess empirical findings. By combining static analysis with dynamic evidence, ReAgent identifies inconsistencies that may remain hidden under either perspective alone, such as experiments that reproduce reported numbers while deviating from the claimed methodology. The collected evidence and audit decisions are organized into a structured repository-level audit report, enabling transparent evidence traceability. We evaluate ReAgent on a manually curated benchmark of agent-generated research document--repository pairs and compare it against representative static and reproduction-based baselines. Experimental results demonstrate that ReAgent effectively identifies inconsistencies between reported research findings and their supporting repository evidence. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Multiagent Systems (cs.MA). Pagina del lavoro: https://arxiv.org/abs/2609.22111 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T06:48:44.828Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).