Metraipolis › archive › dc_7683b48568cb
Verifier-Grounded Agentic RL for Provably Correct Code — arXiv:2610.19298
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.19298
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T14:15:23.637Z
- SHA-256 of the archived text
f6024da3a062953c8fb3d63b78fb253e0cdbb3ac041de0d0e7f02e34b72a5c10- Stable URL of this page
https://metraipolis.com/documento/dc_7683b48568cb
How to cite
Verifier-Grounded Agentic RL for Provably Correct Code — arXiv:2610.19298 — arXiv (en), https://arxiv.org/abs/2610.19298 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7683b48568cb (Metraipolis archive dc_7683b48568cb, sha256 f6024da3a062953c8fb3d63b78fb253e0cdbb3ac041de0d0e7f02e34b72a5c10)
BibTeX
@misc{dc7683b48568cb,
title = {Verifier-Grounded Agentic RL for Provably Correct Code — arXiv:2610.19298},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.19298},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7683b48568cb; sha256 f6024da3a062953c8fb3d63b78fb253e0cdbb3ac041de0d0e7f02e34b72a5c10},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19298. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reinforcement learning for code usually rewards passing tests, which a model can do while remaining wrong on the inputs the tests do not cover. A program verifier instead decides whether a program satisfies a formal specification for all inputs. Language models are almost always paired with verifiers at inference time only, and turning a verifier into a reward raises two difficulties. It reports only acceptance or rejection, which gives no gradient when every attempt fails, and a policy can cheat it, most simply by telling it to assume what the proof was supposed to establish. We build an agentic environment and reward that address both. The reward reads the verifier's diagnostics rather than its verdict, scoring a candidate by the weighted fraction of proof obligations it discharges, extended with hidden probes from a reference proof's annotations. An integrity audit outranks the verifier, rejecting any candidate that alters the specification or assumes what it was meant to prove. The model works as an agent against a live verifier over many turns, with a separately trained proof specialist as a frozen collaborator. Across Qwen3 models from 1.7B to 14B and a 35B Qwen3.6 mixture-of-experts model, held-out verification on code generation rises from 66.1% to 93.5% on DafnyBench and from 56.3% to 100% on SWE-Proof. This reward leads every other training signal we compare, including a binary pass-or-fail reward trained under otherwise identical settings, with the widest margin on code generation. When the agent must also deliver the repository patch, graded by each project's withheld tests, the fraction of held-out issues it resolves rises from 59.6% to 72.5%. Evaluated unchanged on three verification suites withheld from training, the checkpoints improve on all of them, including one whose language and verifier are absent from training. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.19298 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T13:53:35.562Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).