Metraipolis › archive › dc_ee4fe62077f3
Verification Trap: Understanding Test-Time Selection Failures under False Premises in Code Generation — arXiv:2610.05170
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.05170
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:28:17.243Z
- SHA-256 of the archived text
972e40456eed9124acb779e8e59495cd18560c934fce33227d9377ab4c38b9c2- Stable URL of this page
https://metraipolis.com/documento/dc_ee4fe62077f3
How to cite
Verification Trap: Understanding Test-Time Selection Failures under False Premises in Code Generation — arXiv:2610.05170 — arXiv, https://arxiv.org/abs/2610.05170 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ee4fe62077f3 (Metraipolis archive dc_ee4fe62077f3, sha256 972e40456eed9124acb779e8e59495cd18560c934fce33227d9377ab4c38b9c2)
BibTeX
@misc{dcee4fe62077f3,
title = {Verification Trap: Understanding Test-Time Selection Failures under False Premises in Code Generation — arXiv:2610.05170},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.05170},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ee4fe62077f3; sha256 972e40456eed9124acb779e8e59495cd18560c934fce33227d9377ab4c38b9c2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05170. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Test-time compute has become a central way to improve code generation: systems sample multiple candidate programs and use verifier-visible evidence to select the final output. This paradigm implicitly assumes that the verifier provides a corrective signal independent from the generator. We challenge this assumption under misleading task premises. When the generator and verifier share a false premise, they become coupled through a mistaken belief: the generator produces premise-consistent shortcuts, while the verifier supplies evidence that fails to expose them. Consequently, the selector may choose a hidden-test-wrong candidate even when a hidden-test-correct program exists in the pool. We call this failure mode Verification Trap. Across three code-generation benchmarks and five code models, false premises consistently degrade first-sample correctness, reduce selector-chosen correctness after 64-sample test-time selection, and amplify recoverable mis-selection. Mechanistically, verifier-written tests inherit the premise-level blind spot, reshaping verifier-visible candidate space away from hidden-test correctness. These traces make Verification Trap predictable before hidden execution: a lightweight gold-free predictor using verifier-visible features reaches 0.846 AUROC. Our results identify decoupled evidence as a key mitigation axis: coupled scaling provides limited recovery, whereas premise-agnostic robustness auditors recover substantial oracle headroom. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.05170 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:20.313Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).