Metraipolis › archive › dc_bc9c6950ca6c

A Verifier Can Leak the Answer: Diagnosability Before Optimization in Closed-Loop Agent Debugging — arXiv:2610.00126

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.00126
License (as archived)
CC0-1.0
Archived
2026-10-06T11:54:29.956Z
SHA-256 of the archived text
7658022ad2c7954f18fe70f0788ae6f30be4583122e576ac237ff9fe02817c84
Stable URL of this page
https://metraipolis.com/documento/dc_bc9c6950ca6c

How to cite

A Verifier Can Leak the Answer: Diagnosability Before Optimization in Closed-Loop Agent Debugging — arXiv:2610.00126 — arXiv (en), https://arxiv.org/abs/2610.00126 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bc9c6950ca6c (Metraipolis archive dc_bc9c6950ca6c, sha256 7658022ad2c7954f18fe70f0788ae6f30be4583122e576ac237ff9fe02817c84)
BibTeX
@misc{dcbc9c6950ca6c,
  title = {A Verifier Can Leak the Answer: Diagnosability Before Optimization in Closed-Loop Agent Debugging — arXiv:2610.00126},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.00126},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bc9c6950ca6c; sha256 7658022ad2c7954f18fe70f0788ae6f30be4583122e576ac237ff9fe02817c84},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.00126. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Agent developers increasingly compare prompts, tools, policies, and diagnosis algorithms through simulator-grounded verifiers. A verifier can nevertheless make a solver comparison vacuous: if its probes or predicates encode the target identity, an exact optimizer may appear effective without resolving any genuine ambiguity. We report such a failure in an aggregate-trace debugger for a closed-loop decision agent. Exact minimum hitting set (MHS) and a propagation-aware greedy method returned identical supports in 12/12 development cases and the same planted-fault recovery in 9/12. A subsequent audit found that exact-anchor predicates produced the planted pair in 9/9 cases. After removing those anchors, overall planted-pair recovery was 8/9; hard-probe singleton pairs nevertheless matched the planted pair in 9/9, and no case retained a nonempty residual conflict family after propagation (0/9). The optimizer was correct, but the verifier had already disclosed the answer. We replace solver-first evaluation with a support-gated verification contract. A clean reference map must first show repeated component exposure; a matched reference/current gate must then establish comparable runtime evidence; only afterward may an independently calibrated signal rule return a detection. In a preregistered heldout comprising 1,440 cases and 21,600 partition rows, 55/72 regime-component units passed the reference gate, 54/55 passed the runtime gate, and stable false admission was 0/20 represented components with a one-sided exact 95% upper bound of 0.1391. Within admitted units, affected clean traffic predicted detection better than nominal fault-cell fraction. The main lesson is structural: verify evidence eligibility and non-revelation before optimizing the component selector. Otherwise a stronger solver can merely certify a stronger verifier artifact.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.MA).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA).
Pagina del lavoro: https://arxiv.org/abs/2610.00126 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:17:30.552Z dalla pagina https://arxiv.org/catchup/cs.MA/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).