Metraipolis › archive › dc_0841d85e7df8
How Reliable Are NVD CWE Labels? A Large-Scale Semantic Audit with Seclometry — arXiv:2608.21977
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.21977
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T08:12:58.310Z
- SHA-256 of the archived text
218c58886d1ccef7b4cc6bd40b03c66e63dd8c712578bcd33e5a631a191feb66- Stable URL of this page
https://metraipolis.com/documento/dc_0841d85e7df8
How to cite
How Reliable Are NVD CWE Labels? A Large-Scale Semantic Audit with Seclometry — arXiv:2608.21977 — arXiv (en), https://arxiv.org/abs/2608.21977 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0841d85e7df8 (Metraipolis archive dc_0841d85e7df8, sha256 218c58886d1ccef7b4cc6bd40b03c66e63dd8c712578bcd33e5a631a191feb66)
BibTeX
@misc{dc0841d85e7df8,
title = {How Reliable Are NVD CWE Labels? A Large-Scale Semantic Audit with Seclometry — arXiv:2608.21977},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.21977},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0841d85e7df8; sha256 218c58886d1ccef7b4cc6bd40b03c66e63dd8c712578bcd33e5a631a191feb66},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.21977. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
CWE labels in the National Vulnerability Database (NVD) are widely treated as ground truth for vulnerability search, scanner evaluation, benchmark construction, learning-based security tools, and vulnerability prioritization. Yet their reliability has not been systematically measured at scale, despite growing concerns about NVD's enrichment backlog and anecdotal reports of inaccurate, ambiguous, or missing labels. This paper presents a large-scale, code-semantics-grounded measurement of CWE labeling quality in NVD. We build CWEAgent, a validated auditing instrument based on seclometry, a structured representation of vulnerability semantics that captures the root cause, trigger condition, violated security property, exploit mechanism, and impact of vulnerable code. On a manually curated benchmark of 100 open-source CVEs, CWEAgent achieves 85% top-1 accuracy and 92% ambiguity-aware accuracy. Applying CWEAgent to 15,556 open-source CVEs disclosed from 2017-2026, we find that only 49.70% of NVD CWE labels exactly match the code-grounded label. Another 31.37% are defensible alternatives under taxonomy ambiguity, while 3.63% are evidence-inconsistent likely errors. Label reliability varies sharply by assigning organization and weakness type, and apparent project- or language-level differences are largely composition effects of those underlying weakness types. Evidence-inconsistent labels have also increased over time. Through manual review of 434 confirmed mislabels, we identify six recurring error patterns, showing that CWE noise is a structural problem in vulnerability metadata rather than isolated annotation mistakes. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-25 in cs.SE). Categorie arXiv: Cryptography and Security (cs.CR) ; Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2608.21977 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T08:07:33.027Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).