Metraipolis › archive › dc_e50814b1fe06

Verification-Gated Redraw and Repair for Local Small Language Models: A Paired Evaluation — arXiv:2611.00707

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.00707
License (as archived)
CC0-1.0
Archived
2026-11-03T11:27:15.886Z
SHA-256 of the archived text
6bdc9e9059a8c4a76c79a2c95fd215d80c9d1b902062dce71fe96998d258305c
Stable URL of this page
https://metraipolis.com/documento/dc_e50814b1fe06

How to cite

Verification-Gated Redraw and Repair for Local Small Language Models: A Paired Evaluation — arXiv:2611.00707 — arXiv (en), https://arxiv.org/abs/2611.00707 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e50814b1fe06 (Metraipolis archive dc_e50814b1fe06, sha256 6bdc9e9059a8c4a76c79a2c95fd215d80c9d1b902062dce71fe96998d258305c)
BibTeX
@misc{dce50814b1fe06,
  title = {Verification-Gated Redraw and Repair for Local Small Language Models: A Paired Evaluation — arXiv:2611.00707},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00707},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e50814b1fe06; sha256 6bdc9e9059a8c4a76c79a2c95fd215d80c9d1b902062dce71fe96998d258305c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00707. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Executable checks can reject incorrect model-generated programs, but rejection alone does not determine whether fresh sampling or diagnostic repair is preferable. We present a controlled, resource-bounded conceptual replication and extension of established recovery policies: a single answer (N), one verification-triggered redraw (R), and one diagnostic repair (D), for Ornith 1.5 9B and Gemma 4 E4B served locally in nominal 4-bit MLX format. The frozen evaluation contains 72 tasks: 40 HumanEval+ problems, 16 SQL tasks and 16 shell tasks. Policies share initial responses and use only public checks for online decisions; every candidate is model-generated. Each task has one stochastic trajectory per model, so the study does not estimate variation across independent sampling blocks. Redraw increases observed final-test success from 51.4% to 59.7% for Ornith and from 55.6% to 62.5% for Gemma. Diagnostic repair reaches 52.8% and 63.9%, respectively. None of four internally prespecified paired contrasts is significant after Holm correction (minimum adjusted p = 0.125). Public checks falsely accept 25 Ornith and 11 Gemma candidates, demonstrating that acceptance is not a correctness guarantee. A post-review audit of initial answers gives perfect-retry upper bounds of 76.4% and 87.5%, respectively: accepted initial errors cannot trigger recovery under the fixed gate. This one-block, resource-bounded study provides a finite-cohort experimental record, not policy equivalence, a general model ranking or universal reliability

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.SE, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2611.00707 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T11:02:52.181Z dalla pagina https://arxiv.org/list/cs.SE/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).