Metraipolis › archive › dc_a40dbfac1ab0

The Data Problem in Software Vulnerability Analysis: Artifacts, Quality, and Consumption — arXiv:2609.01503

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.01503
License (as archived)
CC0-1.0
Archived
2026-10-17T06:57:08.021Z
SHA-256 of the archived text
e7710a3f9c618f4c410be4141dc31c8d3b956afa4b00d9b80fa8e56c4917af9d
Stable URL of this page
https://metraipolis.com/documento/dc_a40dbfac1ab0

How to cite

The Data Problem in Software Vulnerability Analysis: Artifacts, Quality, and Consumption — arXiv:2609.01503 — arXiv (en), https://arxiv.org/abs/2609.01503 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a40dbfac1ab0 (Metraipolis archive dc_a40dbfac1ab0, sha256 e7710a3f9c618f4c410be4141dc31c8d3b956afa4b00d9b80fa8e56c4917af9d)
BibTeX
@misc{dca40dbfac1ab0,
  title = {The Data Problem in Software Vulnerability Analysis: Artifacts, Quality, and Consumption — arXiv:2609.01503},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.01503},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a40dbfac1ab0; sha256 e7710a3f9c618f4c410be4141dc31c8d3b956afa4b00d9b80fa8e56c4917af9d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.01503. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Learning- and LLM-based software vulnerability analysis is only as trustworthy as the data it is trained and evaluated on, yet that data is rarely examined as a first-class object. We investigate the data behind vulnerability analysis through a dataset-centric taxonomy that separates what an artifact is (code, metadata, patches, tests/PoCs, reasoning, traces), how good it is (realism, label evidence, scale, diversity, leakage, availability), and what it is used for. From a systematically assembled corpus of 1522 papers covering 2016-2026 plus foundational earlier work we deep-code a tiered set of 111 anchor papers, backing every affirmative rubric-graded value with a verbatim span, and we report, per attribute, both how much it has been studied and how well datasets achieve it. The results trace an evidence ladder: executable artifacts are the only major type where 15 of the 24 datasets are both graded real-world and carry labels that received an independent check, while code-sample datasets-the largest category in both the auto-tagged corpus and the anchor set-are the least realistic: 20 of the 41 draw their vulnerabilities from authentic projects or CVEs, but only 3 keep the sample at the unit the code is deployed in, and only 2 do both-though these are coarse component tests, and just one code-sample dataset meets the codebook's stricter full-context real-world grade. Among these, leakage goes unaddressed by 49 of the 90 datasets where it applies, more than a quarter say nothing about availability, reasoning data has arrived only recently and is mostly model-generated, and primary trace corpora remain limited to three datasets, the total after a corpus-wide screen and a full-text check of every candidate it surfaced, with further datasets releasing traces secondarily behind benchmarks and harnesses.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-02 in cs.SE).
Categorie arXiv: Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2609.01503 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T06:43:00.620Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).