Metraipolis › archive › dc_072ee12a91e5

Prior Audit-Repair Context Shifts LLM Verifier Thresholds Toward Leniency — arXiv:2608.16003

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.16003
License (as archived)
CC0-1.0
Archived
2026-10-18T17:41:55.827Z
SHA-256 of the archived text
9a3598b854671d51f47bfacc09f380d92371b70c4f17760b6cadc2c178a6927e
Stable URL of this page
https://metraipolis.com/documento/dc_072ee12a91e5

How to cite

Prior Audit-Repair Context Shifts LLM Verifier Thresholds Toward Leniency — arXiv:2608.16003 — arXiv (en), https://arxiv.org/abs/2608.16003 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_072ee12a91e5 (Metraipolis archive dc_072ee12a91e5, sha256 9a3598b854671d51f47bfacc09f380d92371b70c4f17760b6cadc2c178a6927e)
BibTeX
@misc{dc072ee12a91e5,
  title = {Prior Audit-Repair Context Shifts LLM Verifier Thresholds Toward Leniency — arXiv:2608.16003},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.16003},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_072ee12a91e5; sha256 9a3598b854671d51f47bfacc09f380d92371b70c4f17760b6cadc2c178a6927e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.16003. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Automated checking pipelines increasingly place one language model as the checker and another (or the same one) as the fixer. We ask whether that wiring changes what the checker reports. Measuring false alarms on human-verified-correct ProcessBench traces with the present task held byte-identical, we find that a completed audit -> repair episode already in the model's context lowers false alarms in 15 of 15 model x wording combinations, by 2.8 to 11.5 percentage points against a length-matched non-audit control, a 9 to 25% reduction relative to that control. The direction contradicts what the accumulated-message literature predicts: an episode whose audit reported an error lowers false alarms further still, at all five wordings on the model where that manipulation lands cleanly, though a negativity asymmetry predicts more flagging. Decomposing the episode finds repair content and audit verdict complementary: different components carry the effect on different model families. Signal-detection analysis locates the change in the threshold rather than in discrimination -- the criterion moves in 15 of 15 combinations and survives correction in 13 while d' survives in none, though the d' test is half as sensitive by construction -- and a hand audit of 50 false alarms finds 82% simply wrong, so at this operating point the shift need not be harmful. With reasoning enabled the effect keeps its relative size on both models tested, and the threshold reading holds there too.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.16003 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T16:29:13.847Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).