Metraipolis › archive › dc_aba7fbbbf1b8
AI Safety via Debate is Compromised by Cognitive Biases — arXiv:2610.05461
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.05461
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:25:10.947Z
- SHA-256 of the archived text
56c31c21f66ad93100dab9d93bedc7ad5f9a07f29693a067e661f9e6953a5bb9- Stable URL of this page
https://metraipolis.com/documento/dc_aba7fbbbf1b8
How to cite
AI Safety via Debate is Compromised by Cognitive Biases — arXiv:2610.05461 — arXiv, https://arxiv.org/abs/2610.05461 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aba7fbbbf1b8 (Metraipolis archive dc_aba7fbbbf1b8, sha256 56c31c21f66ad93100dab9d93bedc7ad5f9a07f29693a067e661f9e6953a5bb9)
BibTeX
@misc{dcaba7fbbbf1b8,
title = {AI Safety via Debate is Compromised by Cognitive Biases — arXiv:2610.05461},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.05461},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aba7fbbbf1b8; sha256 56c31c21f66ad93100dab9d93bedc7ad5f9a07f29693a067e661f9e6953a5bb9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05461. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reinforcement learning from human feedback (RLHF) has played a central role in making large language models responsive to human instructions. However, human evaluators often favor flattering or persuasive responses over truthful ones, creating incentives for models to appeal to evaluators at the expense of accuracy. AI safety via debate has been proposed as a way to improve the supervision of language models: in this paradigm, two agents argue opposing positions and challenge each other's claims, potentially exposing falsehoods to the adjudicator. A central premise of AI safety via debate is that truthful arguments are easier to defend than false ones under adversarial scrutiny. In this work, we investigate whether this advantage persists when debaters use rhetorical strategies that exploit biases in human judgment. Inspired by competitive debate, we construct 68 LLM-generated dialogues about detective mysteries with known culprits, spanning four interventions: anchoring, fallacy oversight, pro-jargon, and verbosity. We apply each intervention to either the side advocating for the true culprit or the side advocating for an innocent suspect, allowing us to distinguish influence on adjudication from correctness. In a study with 369 participants, we find that, pooled across bias types, these interventions significantly shift judgments toward the manipulated side. These findings expose a vulnerability in debate-based supervision: human adjudication is sensitive to manipulative rhetorical strategies. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.05461 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:05.590Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).