Metraipolis › archive › dc_ea82d2c610de

Benchmarking Automated Security Patch Backporting: How Far Are We? — arXiv:2608.17671

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.17671
License (as archived)
CC0-1.0
Archived
2026-10-18T14:33:38.816Z
SHA-256 of the archived text
388efc52bea6d823728d411e599ffbd137539f2677868b5619e7595b913b9932
Stable URL of this page
https://metraipolis.com/documento/dc_ea82d2c610de

How to cite

Benchmarking Automated Security Patch Backporting: How Far Are We? — arXiv:2608.17671 — arXiv (en), https://arxiv.org/abs/2608.17671 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ea82d2c610de (Metraipolis archive dc_ea82d2c610de, sha256 388efc52bea6d823728d411e599ffbd137539f2677868b5619e7595b913b9932)
BibTeX
@misc{dcea82d2c610de,
  title = {Benchmarking Automated Security Patch Backporting: How Far Are We? — arXiv:2608.17671},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.17671},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ea82d2c610de; sha256 388efc52bea6d823728d411e599ffbd137539f2677868b5619e7595b913b9932},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17671. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Automated security patch backporting is critical for mitigating N-day vulnerabilities. Recent tools report success rates above 80% on their respective datasets. However, these evaluations are often confined to homogeneous environments, such as one repository or specific project versions. Consequently, it remains unclear how well these tools generalize beyond their originally targeted scenarios. We present Porting Benchmark, a curated dataset of 1,234 security patch backporting cases spanning cross-version, cross-branch, and cross-repository scenarios, paired with a common evaluation framework. Using this benchmark, we evaluate five tools spanning program analysis, LLM prompting, and LLM agents under aligned settings. Our results show that aligned evaluation changes the apparent performance landscape: PortGPT and TSBPort remain comparatively strong on the Replication Dataset, while FixMorph and Mystique degrade substantially under the common protocol. Performance degrades sharply on structurally complex patches: the best commit-level success rate falls from 85.2% on Type-I patches to 24.0% on Type-IV. We identify four root-cause categories (missing target API awareness, cross-version semantic mismatch, non-local dependency propagation failure, and patch construction or localization failure) and derive concrete directions for next-generation tool design. On a 45-case dynamically validated subset with verified test cases and constructed POCs, we further observe that reference-based benchmark scores do not fully capture real-world remediation: exact match sharply under-credits harder target adaptations, while executable validation reveals residual integration failures in the target that static reference agreement misses. Executable-feedback refinement provides limited but measurable recovery on the hardest executable cases.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.AI).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2608.17671 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T14:27:54.449Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).