Metraipolis › archive › dc_da0700fc47d7
Groundability, Not Scale Alone: When Weak Reviewers Can Audit Strong Coding Agents — arXiv:2610.01023
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.01023
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T12:12:25.670Z
- SHA-256 of the archived text
8408cc2013b92a641187b84f0901d0a4b5f3b1e15c3b4d47509ac2fbeca34b39- Stable URL of this page
https://metraipolis.com/documento/dc_da0700fc47d7
How to cite
Groundability, Not Scale Alone: When Weak Reviewers Can Audit Strong Coding Agents — arXiv:2610.01023 — arXiv (en), https://arxiv.org/abs/2610.01023 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_da0700fc47d7 (Metraipolis archive dc_da0700fc47d7, sha256 8408cc2013b92a641187b84f0901d0a4b5f3b1e15c3b4d47509ac2fbeca34b39)
BibTeX
@misc{dcda0700fc47d7,
title = {Groundability, Not Scale Alone: When Weak Reviewers Can Audit Strong Coding Agents — arXiv:2610.01023},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.01023},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_da0700fc47d7; sha256 8408cc2013b92a641187b84f0901d0a4b5f3b1e15c3b4d47509ac2fbeca34b39},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01023. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Coding agents can return plausible patches that omit required behavior. These failures are hard to review because long traces and confident summaries often hide what was missed. We ask when a nominally weaker reviewer can reliably decide whether a patch solves its issue. We study 411 execution-labeled traces from three agents and 101 controlled cases. On 154 GPT-5.4 traces, structured but unchecked evidence raises both defect catch and over-rejection. We then provide official execution evidence as an upper-bound diagnostic. After choosing and freezing one of two formats per reviewer, five of six reviewers improve both rates on 122 held-out traces; two classify every trace correctly. Reviewer size is not a consistent predictor of quality. Because official tests are unavailable in deployment, we also evaluate a frozen cascade with patch-caused static errors and generated tests that first fail on the unpatched repository. On 121 scored held-out GPT-5.4 traces and 59 Gemini traces, its coverage is 0.89 and 0.86, risk is 0.33 and 0.26, catch is 0.76 and 0.80, and over-rejection is 0.66 and 0.67. Most false rejections occur when unresolved cases reach the reviewer. Official execution evidence shows the potential of weak review when decisive checks are available. Producing equally reliable checks without official tests remains the main bottleneck. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI). Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.01023 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T10:21:28.145Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).