Metraipolis › archive › dc_bc9c6950ca6c
A Verifier Can Leak the Answer: Diagnosability Before Optimization in Closed-Loop Agent Debugging — arXiv:2610.00126
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.00126
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T11:54:29.956Z
- SHA-256 of the archived text
7658022ad2c7954f18fe70f0788ae6f30be4583122e576ac237ff9fe02817c84- Stable URL of this page
https://metraipolis.com/documento/dc_bc9c6950ca6c
How to cite
A Verifier Can Leak the Answer: Diagnosability Before Optimization in Closed-Loop Agent Debugging — arXiv:2610.00126 — arXiv (en), https://arxiv.org/abs/2610.00126 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bc9c6950ca6c (Metraipolis archive dc_bc9c6950ca6c, sha256 7658022ad2c7954f18fe70f0788ae6f30be4583122e576ac237ff9fe02817c84)
BibTeX
@misc{dcbc9c6950ca6c,
title = {A Verifier Can Leak the Answer: Diagnosability Before Optimization in Closed-Loop Agent Debugging — arXiv:2610.00126},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.00126},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bc9c6950ca6c; sha256 7658022ad2c7954f18fe70f0788ae6f30be4583122e576ac237ff9fe02817c84},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.00126. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Agent developers increasingly compare prompts, tools, policies, and diagnosis algorithms through simulator-grounded verifiers. A verifier can nevertheless make a solver comparison vacuous: if its probes or predicates encode the target identity, an exact optimizer may appear effective without resolving any genuine ambiguity. We report such a failure in an aggregate-trace debugger for a closed-loop decision agent. Exact minimum hitting set (MHS) and a propagation-aware greedy method returned identical supports in 12/12 development cases and the same planted-fault recovery in 9/12. A subsequent audit found that exact-anchor predicates produced the planted pair in 9/9 cases. After removing those anchors, overall planted-pair recovery was 8/9; hard-probe singleton pairs nevertheless matched the planted pair in 9/9, and no case retained a nonempty residual conflict family after propagation (0/9). The optimizer was correct, but the verifier had already disclosed the answer. We replace solver-first evaluation with a support-gated verification contract. A clean reference map must first show repeated component exposure; a matched reference/current gate must then establish comparable runtime evidence; only afterward may an independently calibrated signal rule return a detection. In a preregistered heldout comprising 1,440 cases and 21,600 partition rows, 55/72 regime-component units passed the reference gate, 54/55 passed the runtime gate, and stable false admission was 0/20 represented components with a one-sided exact 95% upper bound of 0.1391. Within admitted units, affected clean traffic predicted detection better than nominal fault-cell fraction. The main lesson is structural: verify evidence eligibility and non-revelation before optimizing the component selector. Otherwise a stronger solver can merely certify a stronger verifier artifact. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.MA). Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA). Pagina del lavoro: https://arxiv.org/abs/2610.00126 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T10:17:30.552Z dalla pagina https://arxiv.org/catchup/cs.MA/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).