Metraipolis › archive › dc_2bdb0243d652

MetaVul: Agentic Harness Discovery for Vulnerability Detection — arXiv:2610.40721

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.40721
License (as archived): CC0-1.0
Archived: 2026-11-03T08:02:28.033Z
SHA-256 of the archived text: 31ede60ae50d0e7e44a00c27c4b93c6265238260c5b1180031b02ab78ce6f25e
Stable URL of this page: https://metraipolis.com/documento/dc_2bdb0243d652

How to cite

MetaVul: Agentic Harness Discovery for Vulnerability Detection — arXiv:2610.40721 — arXiv (en), https://arxiv.org/abs/2610.40721 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2bdb0243d652 (Metraipolis archive dc_2bdb0243d652, sha256 31ede60ae50d0e7e44a00c27c4b93c6265238260c5b1180031b02ab78ce6f25e)

BibTeX

@misc{dc2bdb0243d652,
  title = {MetaVul: Agentic Harness Discovery for Vulnerability Detection — arXiv:2610.40721},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.40721},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2bdb0243d652; sha256 31ede60ae50d0e7e44a00c27c4b93c6265238260c5b1180031b02ab78ce6f25e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.40721. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLM agents are increasingly used to automate vulnerability detection, but reliable detection is challenging because agents must distinguish subtle, security-relevant fixes from benign code variation. While underlying LLM capability has improved significantly, the accuracy of the agent still depends heavily on the harness: the code that defines the scaffolding around the LLM. Harnesses are typically fixed, and largely hand-written, limiting adaptation to evaluator feedback. Additionally, when harnesses are static, improvements on one dataset or model may not transfer to others. To address these challenges, we introduce MetaVul, a self-improving framework for harness discovery in vulnerability detection. Starting from a seed structured reasoning scaffold, MetaVul uses a workflow-based optimization loop to iteratively refine the harness using feedback from paired vulnerable and patched functions, while preserving the structured reasoning and verdict format. At each iteration, it searches over the history of prior harnesses, with access to their trajectories, logs, and metrics, and it returns a set of Pareto-optimal harnesses. We apply MetaVul to discover agent harnesses for various architectures. A validation set (202 pairs) is used for optimization, while an unseen dataset (430 pairs) is kept as the held-out set. We test whether harnesses learned from one model transfer to others not seen during optimization. Results on both frontier and small LLMs show that the optimized harnesses improve pairwise vulnerability detection up to 23.3 percentage points on the validation set and up to 25.8 points on the held-out set with a Pareto-frontier harness. Harnesses learned with one model also transfer to others. We show that harnesses optimized with one model transfer to another, improving held-out pairwise accuracy by 14.7 points without further optimization.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2610.40721 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T07:59:38.761Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).