Metraipolis › archive › dc_5d326087d2fd
Analysis of Local Open-Weight LLM Agents on Vulnerability-Reproduction Tasks — arXiv:2611.06257
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.06257
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-05T18:21:06.333Z
- SHA-256 of the archived text
b226fc4523d48b1d01f2651f64481d406fb3a44d6650e76ef7b6e2b32250128d- Stable URL of this page
https://metraipolis.com/documento/dc_5d326087d2fd
How to cite
Analysis of Local Open-Weight LLM Agents on Vulnerability-Reproduction Tasks — arXiv:2611.06257 — arXiv (en), https://arxiv.org/abs/2611.06257 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5d326087d2fd (Metraipolis archive dc_5d326087d2fd, sha256 b226fc4523d48b1d01f2651f64481d406fb3a44d6650e76ef7b6e2b32250128d)
BibTeX
@misc{dc5d326087d2fd,
title = {Analysis of Local Open-Weight LLM Agents on Vulnerability-Reproduction Tasks — arXiv:2611.06257},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.06257},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5d326087d2fd; sha256 b226fc4523d48b1d01f2651f64481d406fb3a44d6650e76ef7b6e2b32250128d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.06257. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Open-weight language models that fit on a single consumer GPU are becoming capable on a growing range of tasks, and they matter because they can run locally without sending the code to a hosted service. Vulnerability reproduction, turning a vulnerability description into an input that crashes the vulnerable build, is a natural target for them, since it is an automated analysis task closer to test generation than to code comprehension and it runs against a build that must be executed rather than read. In order to assess whether current models can perform it, we evaluate twelve configurations of local models, spanning three generations, two architectures and three quantization levels, on ten CyberGym vulnerability-reproduction tasks, three runs each under one harness, and release the full artefact. We show that every configuration, down to the 4B model, solves most of the six easier tasks, and configurations separate only on the remaining four. For the harder tasks, the 27B model is required. Quantization did not change what the models solved in any matched pair, but it changed how fast they got there, and the mixture of experts reached the same scores in less wall clock. We also examine the agent trajectories. Almost every run we examined, including those that fail, names the vulnerable function and states the correct hypothesis, and what separates a solved task from an unsolved one is whether that hypothesis becomes a working input. Tema degli esploratori: fiducia digitale e crittografia (elenco arXiv cs.CR, nuovi lavori del giorno). Categorie arXiv: Cryptography and Security (cs.CR) ; Machine Learning (cs.LG); Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2611.06257 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-05T18:15:18.888Z dalla pagina https://arxiv.org/list/cs.CR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).