Metraipolis › archive › dc_b2c1efbe514a
Specification-Driven Benchmarking for Automated Program Repair From Static Corpora to Executable Specifications — arXiv:2609.28896
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.28896
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T06:01:48.340Z
- SHA-256 of the archived text
13e72cb6982b5917823125a3887a9a40633c16f9f39b62814cf25aa329a1f029- Stable URL of this page
https://metraipolis.com/documento/dc_b2c1efbe514a
How to cite
Specification-Driven Benchmarking for Automated Program Repair From Static Corpora to Executable Specifications — arXiv:2609.28896 — arXiv (en), https://arxiv.org/abs/2609.28896 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b2c1efbe514a (Metraipolis archive dc_b2c1efbe514a, sha256 13e72cb6982b5917823125a3887a9a40633c16f9f39b62814cf25aa329a1f029)
BibTeX
@misc{dcb2c1efbe514a,
title = {Specification-Driven Benchmarking for Automated Program Repair From Static Corpora to Executable Specifications — arXiv:2609.28896},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.28896},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b2c1efbe514a; sha256 13e72cb6982b5917823125a3887a9a40633c16f9f39b62814cf25aa329a1f029},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.28896. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Automated Program Repair (APR) benchmarks have traditionally been constructed as static datasets whose characteristics are inherited from the defects they contain. While this paradigm has enabled decades of progress, finite corpora provide limited experimental control, become increasingly susceptible to contamination as they are reused, and cannot be systematically regenerated or adapted as evaluation requirements evolve. We propose specification-driven benchmarking, a paradigm in which benchmarks are defined by executable specifications and realized through benchmark generation. The specification explicitly declares the intended properties of the benchmark (including program context, fault taxonomy, difficulty, validation strategy, and corpus constraints) while a generation pipeline realizes those requirements through independent generation, validation, and corpus management components. We develop the conceptual foundations of this approach by introducing a taxonomy of benchmark specification dimensions, establishing how each specification dimension maps to deterministic architectural responsibilities, and arguing that independent validation is a structural requirement for trustworthy benchmark generation. An end-to-end example illustrates how specification choices propagate through the pipeline to produce benchmark instances whose properties are independently verifiable. By treating the benchmark as an executable specification rather than a static dataset, the proposed paradigm shifts benchmark construction from artifact curation to declarative experimental design. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-25 in cs.SE). Categorie arXiv: Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2609.28896 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T05:25:51.941Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).