Metraipolis › archive › dc_48be53ddae8a

DeflectBench: A Benchmark for Evaluating Rhetorical Fallacy Generation in LLMs — arXiv:2608.26119

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.26119
License (as archived)
CC0-1.0
Archived
2026-10-17T15:17:59.036Z
SHA-256 of the archived text
c5ee207f810494d2d85458e915e38580baa4d8d68b406090b227fb6a6694052e
Stable URL of this page
https://metraipolis.com/documento/dc_48be53ddae8a

How to cite

DeflectBench: A Benchmark for Evaluating Rhetorical Fallacy Generation in LLMs — arXiv:2608.26119 — arXiv (en), https://arxiv.org/abs/2608.26119 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_48be53ddae8a (Metraipolis archive dc_48be53ddae8a, sha256 c5ee207f810494d2d85458e915e38580baa4d8d68b406090b227fb6a6694052e)
BibTeX
@misc{dc48be53ddae8a,
  title = {DeflectBench: A Benchmark for Evaluating Rhetorical Fallacy Generation in LLMs — arXiv:2608.26119},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.26119},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_48be53ddae8a; sha256 c5ee207f810494d2d85458e915e38580baa4d8d68b406090b227fb6a6694052e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.26119. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Whether large language models can be prompted to generate rhetorical fallacies on demand, and whether current safety post-training constrains this behavior, has received less attention than the related question of detecting fallacies in existing text. We close this gap with DeflectBench, evaluating 23,990 generations from four frontier models across three deflection strategies (whataboutism, ad hominem, red herring), seven prompt framings, and 80 claims spanning four controversy levels. Refusal is governed primarily by request structure rather than claim content. Per claim refusal varies by only 11 percentage points across the 80 claims, while a single prompt frame change can swing within model refusal by nearly 100 percentage points and switching the requested fallacy type can swing it by over 80 percentage points within explicit framings. An educational debate coach prompt framing collapses refusal to near zero across all four model families, but the bypassed behavior is not clean compliance. Models typically produce labeled compliance, naming the requested manipulation in the same response that contains it. The four models distribute differently across refusal, labeled compliance, soft refusal, and clean compliance. The code and dataset are released at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.26119 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T15:15:18.009Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).