Metraipolis › archive › dc_ca24b64d5000

MetaVul: Agentic Harness Discovery for Vulnerability Detection — arXiv:2610.40721

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.40721
License (as archived): CC0-1.0
Archived: 2026-11-03T05:32:07.412Z
SHA-256 of the archived text: ce1c5c197d354673767759f77cb7dadf374e5715b07b7bafc2209d031ddf5d2e
Stable URL of this page: https://metraipolis.com/documento/dc_ca24b64d5000

How to cite

MetaVul: Agentic Harness Discovery for Vulnerability Detection — arXiv:2610.40721 — arXiv (en), https://arxiv.org/abs/2610.40721 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ca24b64d5000 (Metraipolis archive dc_ca24b64d5000, sha256 ce1c5c197d354673767759f77cb7dadf374e5715b07b7bafc2209d031ddf5d2e)

BibTeX

@misc{dcca24b64d5000,
  title = {MetaVul: Agentic Harness Discovery for Vulnerability Detection — arXiv:2610.40721},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.40721},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ca24b64d5000; sha256 ce1c5c197d354673767759f77cb7dadf374e5715b07b7bafc2209d031ddf5d2e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.40721. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLM agents are increasingly used to automate vulnerability detection, but reliable detection is challenging because agents must distinguish subtle, security-relevant fixes from benign code variation. While underlying LLM capability has improved significantly, the accuracy of the agent still depends heavily on the harness: the code that defines the scaffolding around the LLM. Harnesses are typically fixed, and largely hand-written, limiting adaptation to evaluator feedback. Additionally, when harnesses are static, improvements on one dataset or model may not transfer to others. To address these challenges, we introduce MetaVul, a self-improving framework for harness discovery in vulnerability detection. Starting from a seed structured reasoning scaffold, MetaVul uses a workflow-based optimization loop to iteratively refine the harness using feedback from paired vulnerable and patched functions, while preserving the structured reasoning and verdict format. At each iteration, it searches over the history of prior harnesses, with access to their trajectories, logs, and metrics, and it returns a set of Pareto-optimal harnesses. We apply MetaVul to discover agent harnesses for various architectures. A validation set (202 pairs) is used for optimization, while an unseen dataset (430 pairs) is kept as the held-out set. We test whether harnesses learned from one model transfer to others not seen during optimization. Results on both frontier and small LLMs show that the optimized harnesses improve pairwise vulnerability detection up to 23.3 percentage points on the validation set and up to 25.8 points on the held-out set with a Pareto-frontier harness. Harnesses learned with one model also transfer to others. We show that harnesses optimized with one model transfer to another, improving held-out pairwise accuracy by 14.7 points without further optimization.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2610.40721 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T05:18:52.168Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).