Metraipolis › archive › dc_889ef50ebe5c

How Do Coding Agents Optimize Software and Report Performance Validation? A Large-Scale Empirical Study of Open-Source Pull Requests — arXiv:2610.03969

Source
arXiv
Original
https://arxiv.org/abs/2610.03969
License (as archived)
CC0-1.0
Archived
2026-10-06T05:29:13.826Z
SHA-256 of the archived text
0f41be58d5168c6186fac5d2ee3236704f53d52573821cb95fcda07614766e65
Stable URL of this page
https://metraipolis.com/documento/dc_889ef50ebe5c

How to cite

How Do Coding Agents Optimize Software and Report Performance Validation? A Large-Scale Empirical Study of Open-Source Pull Requests — arXiv:2610.03969 — arXiv, https://arxiv.org/abs/2610.03969 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_889ef50ebe5c (Metraipolis archive dc_889ef50ebe5c, sha256 0f41be58d5168c6186fac5d2ee3236704f53d52573821cb95fcda07614766e65)
BibTeX
@misc{dc889ef50ebe5c,
  title = {How Do Coding Agents Optimize Software and Report Performance Validation? A Large-Scale Empirical Study of Open-Source Pull Requests — arXiv:2610.03969},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.03969},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_889ef50ebe5c; sha256 0f41be58d5168c6186fac5d2ee3236704f53d52573821cb95fcda07614766e65},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.03969. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Software performance optimization requires identifying bottlenecks and improvement opportunities, and empirically evaluating the effects and trade-offs across performance objectives. Autonomous AI coding agents submit performance-oriented pull requests (PRs) to open-source projects, yet it remains unclear whether these changes reflect established optimization practices and adequately substantiate their claimed value. Extending our MSR 2026 Mining Challenge pilot study of 407 performance PRs, we analyze a sample of 1,130 agentic and 1,130 human-authored performance PRs collected through June 2026. We compare adoption outcomes and patch characteristics between the two groups, and examine their optimization practices and validation behaviors. The results show that agentic PRs are merged less often than human-authored PRs (54.4% vs. 73.3%), but the two groups use similar optimization strategies and report validation at comparable rates. Agentic PRs rely more on static reasoning and less on benchmarks (46.1% vs. 57.2%), although benchmark use converges by the end of the study period. Agentic PRs also report validation for code-smell refactorings as often as for resource-targeting changes, whereas human-authored PRs do so less often. Across both groups, about half of validated PRs report no quantitative performance metric, and relevant trade-offs are rarely quantified. Agentic optimization practice has become increasingly similar to human practice, but the supporting evidence remains incomplete. Establishing whether agent-proposed optimizations are supported by rigorous, quantitative, and multidimensional evidence remains an important challenge. These findings motivate evaluation infrastructure and review criteria that measure intended effects and relevant costs rather than treating the presence of validation alone as sufficient.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.PF, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE) ; Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2610.03969 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:25:05.316Z dalla pagina https://arxiv.org/list/cs.PF/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).