Metraipolis › archive › dc_fd4d0f282962

Black-Box Auditing of Epistemic Reliability in Multi-Agent Debate Distillation — arXiv:2609.32361

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32361
License (as archived)
CC0-1.0
Archived
2026-10-07T21:00:07.194Z
SHA-256 of the archived text
ba74847183c6f77d86517260090d22e23d9cc6338a7108d41e9d0fb428cb5dbc
Stable URL of this page
https://metraipolis.com/documento/dc_fd4d0f282962

How to cite

Black-Box Auditing of Epistemic Reliability in Multi-Agent Debate Distillation — arXiv:2609.32361 — arXiv (en), https://arxiv.org/abs/2609.32361 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fd4d0f282962 (Metraipolis archive dc_fd4d0f282962, sha256 ba74847183c6f77d86517260090d22e23d9cc6338a7108d41e9d0fb428cb5dbc)
BibTeX
@misc{dcfd4d0f282962,
  title = {Black-Box Auditing of Epistemic Reliability in Multi-Agent Debate Distillation — arXiv:2609.32361},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32361},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fd4d0f282962; sha256 ba74847183c6f77d86517260090d22e23d9cc6338a7108d41e9d0fb428cb5dbc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32361. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Debate distillation adapts weaker verifiers using multi-agent debate transcripts to improve their judgement in subsequent debates, but gains on monitored tasks do not establish reliability on related unmonitored tasks. We study epistemic reliability degradation, in which adaptation preserves monitored performance while reducing support for correct responses on hidden tasks. We consider an adversarial debater that manipulates debate arguments while defending the correct monitored response, and ask whether the resulting degradation merely reflects catastrophic forgetting and whether standard evaluation can detect it. To address these questions, we propose ER-Audit, a two-stage black-box auditing framework that compares frozen verifier checkpoints before and after adaptation, and introduce two evaluation benchmarks pairing monitored and hidden task prompts grounded in shared contexts. ER-Audit searches for counterexamples to non-degradation by evaluating semantically valid paraphrases and, if none is found, uses independent paraphrases for sequential hypothesis testing. We derive anytime-valid lower confidence bounds on the non-degradation probability, allowing data-dependent stopping within a finite budget. We further establish a common lower bound across fixed paraphrase distributions and extend it to distributions within a bounded total variation distance of their mixtures. Our experiments show that higher hidden-task accuracy can coexist with more counterexamples to non-degradation and lower non-degradation bounds. This divergence challenges explanations based solely on broad catastrophic forgetting and shows that auditing can uncover selective hidden-task degradation concealed by aggregate performance gains. Our code and benchmarks are available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2609.32361 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:02.308Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).