Metraipolis › archive › dc_589b985e9ea5

SeVerus: Context-Aware, Gamified Benchmarking for Rust Verification — arXiv:2611.07194

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.07194
License (as archived)
CC0-1.0
Archived
2026-11-06T14:42:12.755Z
SHA-256 of the archived text
200f6eb187f007c903d0023b85994950e054a74f207ab218f0c918493e948753
Stable URL of this page
https://metraipolis.com/documento/dc_589b985e9ea5

How to cite

SeVerus: Context-Aware, Gamified Benchmarking for Rust Verification — arXiv:2611.07194 — arXiv (en), https://arxiv.org/abs/2611.07194 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_589b985e9ea5 (Metraipolis archive dc_589b985e9ea5, sha256 200f6eb187f007c903d0023b85994950e054a74f207ab218f0c918493e948753)
BibTeX
@misc{dc589b985e9ea5,
  title = {SeVerus: Context-Aware, Gamified Benchmarking for Rust Verification — arXiv:2611.07194},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.07194},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_589b985e9ea5; sha256 200f6eb187f007c903d0023b85994950e054a74f207ab218f0c918493e948753},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07194. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Coding agents achieve high success rates on existing benchmarks for Verus, a verifier for Rust programs. These benchmarks typically provide the auxiliary lemmas needed to complete a proof. In practice, however, such lemmas must be discovered and proved. To capture this challenge, we present SEVERUS, a context-aware technique to generate Verus benchmarks that organizes verification tasks into quests, each divided into levelsin a gamified manner. Using the original proof dependencies in verified projects, SEVERUS constructs one quest per verification target and progressively removes proof context to create individual levels. These levels range from the easiest, requiring completing only a single missing lemma, to the hardest, requiring proving the target without any supporting lemmas. An independent checker rejects changes to executable code, weakened specifications, and unsound proof constructs. A noisy binary search estimates the highest level that an agent can solve reliably. Applied to six open-source Verus projects, SEVERUS generates 819 quests comprising 10,095 levels. Experiments with Codex using GPT-6 Sol and GPT-6 Luna, and with Claude Code using Sonnet 4.6, show that success decreases and cost increases as proof context is withdrawn. The strongest agent solves the final level of 52 of 60 sampled quests, yet can only reliably solve 35% of their levels, with failures concentrated in longer quests. Successful GPT-6 Sol proofs rarely reproduce the withheld lemmas, instead, using alternative proof decompositions. By systematically varying proof context, SEVERUS provides a foundation for constructing verification benchmarks that can continue to challenge increasingly capable coding agents.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.07194 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:18:42.667Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).