Metraipolis › archive › dc_ecc94db2ca70

Fixed Detectors, Moving Code: How Well Do Static Security Analyzers Hold Up over a Decade of C/C++ Evolution? — arXiv:2611.03742

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.03742
License (as archived): CC0-1.0
Archived: 2026-11-04T18:52:28.111Z
SHA-256 of the archived text: 3f2edfc3c90f93946096f592acd8af6f9310c7789328cdd534d019e4729fa538
Stable URL of this page: https://metraipolis.com/documento/dc_ecc94db2ca70

How to cite

Fixed Detectors, Moving Code: How Well Do Static Security Analyzers Hold Up over a Decade of C/C++ Evolution? — arXiv:2611.03742 — arXiv (en), https://arxiv.org/abs/2611.03742 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ecc94db2ca70 (Metraipolis archive dc_ecc94db2ca70, sha256 3f2edfc3c90f93946096f592acd8af6f9310c7789328cdd534d019e4729fa538)

BibTeX

@misc{dcecc94db2ca70,
  title = {Fixed Detectors, Moving Code: How Well Do Static Security Analyzers Hold Up over a Decade of C/C++ Evolution? — arXiv:2611.03742},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03742},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ecc94db2ca70; sha256 3f2edfc3c90f93946096f592acd8af6f9310c7789328cdd534d019e4729fa538},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03742. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Static application security testing (SAST) tools are often deployed on the assumption that detectors configured today will remain effective on future code, including previously unknown vulnerabilities. We test this assumption using 20,000 vulnerable and fixed C/C++ cases from 1,116 open-source projects (2016-2025), evaluated with three CodeQL releases, three Infer releases, and one Semgrep configuration. To distinguish temporal effects from shifts in code composition, we compare yearly populations, project- and label-matched year pairs, and within-project trajectories. We investigate the largest within-project declines through repair diffs, detection predicates, alert locations, and execution logs, while accounting for failed, degraded, and costly analyses. Under changed-file case-label scoring, overall F1 ranges from 0.066 to 0.478, but count-based AUROC stays near chance (0.495-0.526). Newer CodeQL releases attain higher F1 alongside nearly parallel rises in recall and false-positive rate, not stronger separation of vulnerable from fixed cases. In project-matched year pairs, CodeQL R2024 has negative mean later-minus-earlier F1 at every source-year gap, whereas Infer v1.1 and v1.3 have positive means throughout. Among selected drops, only 6 of 39 CodeQL and 2 of 620 Infer findings overlap a changed repair line. For Semgrep, 78.96% of cases show parser or timeout diagnostics even though every run reports success. These results suggest that apparent changes in temporal SAST effectiveness can arise from broader alerting, changing analysis coverage, and incidental findings, not solely from losses in vulnerability detection. Evaluations of temporal sustainability should fix detector conditions, control for projects, inspect repair-level evidence, and report failed or degraded analyses.

Tema degli esploratori: fiducia digitale e crittografia (elenco arXiv cs.CR, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR) ; Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2611.03742 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T18:35:07.108Z dalla pagina https://arxiv.org/list/cs.CR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).