Metraipolis › archive › dc_71189f397283
Not All Agents Are Equal: Code Quality and Post-Merge Maintenance Across Five Autonomous Coding Agents in the Wild — arXiv:2609.17598
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.17598
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T15:38:48.158Z
- SHA-256 of the archived text
5964f2c8ad778ed0dffdae15f4d459dea8f1d0525a3c2900aaa0b0a5b534c78a- Stable URL of this page
https://metraipolis.com/documento/dc_71189f397283
How to cite
Not All Agents Are Equal: Code Quality and Post-Merge Maintenance Across Five Autonomous Coding Agents in the Wild — arXiv:2609.17598 — arXiv (en), https://arxiv.org/abs/2609.17598 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_71189f397283 (Metraipolis archive dc_71189f397283, sha256 5964f2c8ad778ed0dffdae15f4d459dea8f1d0525a3c2900aaa0b0a5b534c78a)
BibTeX
@misc{dc71189f397283,
title = {Not All Agents Are Equal: Code Quality and Post-Merge Maintenance Across Five Autonomous Coding Agents in the Wild — arXiv:2609.17598},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.17598},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_71189f397283; sha256 5964f2c8ad778ed0dffdae15f4d459dea8f1d0525a3c2900aaa0b0a5b534c78a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17598. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Autonomous coding agents now open pull requests in public repositories at a scale that was out of reach two years ago, yet little is known about what happens to that code after it lands. This paper studies 37,623 provenance-labeled pull requests (PRs) from five commercial agents (OpenAI Codex, Devin, GitHub Copilot, Cursor, and Claude Code) and a matched human baseline, drawn from 2,807 GitHub repositories between December 2024 and July 2025. We combine the AIDev dataset with 58,792 cached GitHub API responses to measure security smells in added code, structural maintainability, post-merge churn, revert rates, and human review behavior. Three results stand out. First, quality differences are vendor-specific rather than uniform: Codex-authored PRs were reverted about half as often as human PRs (6.1% vs. 11.5%, odds ratio 0.50), while Devin PRs were reverted more often (14.5%, odds ratio 1.31). Second, agent code pooled across vendors was less likely than human code to contain a security smell (odds ratio 0.63), driven by fewer hardcoded credentials and eval-style constructs. Third, review effort concentrates unevenly: Copilot PRs drew the most human reviews and change requests, and Claude Code PRs waited the longest for a first human review (median 12.6 hours). All pipeline code, statistical reports, and figures are released for replication. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-17 in cs.SE). Categorie arXiv: Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2609.17598 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T15:04:30.914Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).