Metraipolis › archive › dc_617f69710f42
Finding the Pivots to Repair Broken Source-Test Evolution Using LLMs — arXiv:2611.03197
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.03197
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T18:41:43.857Z
- SHA-256 of the archived text
8d0b1ac2fc12b59d4c5fbeafc2840d956f00d0b5e041afc61e686829c9ceed0f- Stable URL of this page
https://metraipolis.com/documento/dc_617f69710f42
How to cite
Finding the Pivots to Repair Broken Source-Test Evolution Using LLMs — arXiv:2611.03197 — arXiv (en), https://arxiv.org/abs/2611.03197 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_617f69710f42 (Metraipolis archive dc_617f69710f42, sha256 8d0b1ac2fc12b59d4c5fbeafc2840d956f00d0b5e041afc61e686829c9ceed0f)
BibTeX
@misc{dc617f69710f42,
title = {Finding the Pivots to Repair Broken Source-Test Evolution Using LLMs — arXiv:2611.03197},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.03197},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_617f69710f42; sha256 8d0b1ac2fc12b59d4c5fbeafc2840d956f00d0b5e041afc61e686829c9ceed0f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03197. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Repairing obsolete tests in response to source code evolution is critical for test suite reliability and software quality assurance. While large language models (LLMs) have emerged as promising tools for automating many software development tasks, the effective supply of context and the orchestration of repair pipelines in this source--test co-evolution scenario remain poorly understood. Moreover, although reasoning is found helpful for LLMs on some complex coding tasks, it remains unclear which reasoning steps actually contribute to effective test repairs and how the insights learned from LLMs' behaviors benefit coding agents. In this paper, we present a systematic empirical study of test repair for source-test co-evolution empowered by LLMs, investigating the effective setups and typical LLM behaviors. To support this study, we construct a new benchmark of 500 real-world source-test co-evolution cases from commits submitted between May 2024 and April 2026 across 15 highly-starred open-source Java projects. Our experimental results reveal several key insights for effective LLM-based test repair: the complementary benefits of combining test failure symptoms, source code changes, and supportive code context, as well as the effectiveness of self-repair rounds. We further derive a common reasoning action taxonomy from LLMs' reasoning traces, which proves instrumental in guiding LLMs to repair tests more effectively while at a lower financial cost. Applying our findings to the Codex agent further improves its repair success rate on challenging cases by 66.7%. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.SE, nuovi lavori del giorno). Categorie arXiv: Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2611.03197 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T18:13:07.761Z dalla pagina https://arxiv.org/list/cs.SE/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).