Metraipolis › archive › dc_56663fc7df62

Accounting for Stochasticity in Studies of Large Language Model Refusal — arXiv:2609.33743

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33743
License (as archived)
CC0-1.0
Archived
2026-10-07T22:35:31.358Z
SHA-256 of the archived text
3d3b5aab7c61afe09414faaa043d4a828f50dc8ff5d9fe0cfa51c82376a5e8c0
Stable URL of this page
https://metraipolis.com/documento/dc_56663fc7df62

How to cite

Accounting for Stochasticity in Studies of Large Language Model Refusal — arXiv:2609.33743 — arXiv (en), https://arxiv.org/abs/2609.33743 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_56663fc7df62 (Metraipolis archive dc_56663fc7df62, sha256 3d3b5aab7c61afe09414faaa043d4a828f50dc8ff5d9fe0cfa51c82376a5e8c0)
BibTeX
@misc{dc56663fc7df62,
  title = {Accounting for Stochasticity in Studies of Large Language Model Refusal — arXiv:2609.33743},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33743},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_56663fc7df62; sha256 3d3b5aab7c61afe09414faaa043d4a828f50dc8ff5d9fe0cfa51c82376a5e8c0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33743. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We present preliminary empirical evidence that single-observation queries are insufficient for evaluations of LLM refusal behaviors. Using a longitudinal auditing system, we issued identical prompts 100 times each across four dates to GPT-4.1 for two socially salient topics across 20 Wikipedia sources. Refusal outcomes were consistent with a stable Bernoulli process, yet 20\% of sources fell within a decision-boundary region where a single query is largely uninformative. Reliable quantification of refusals required between 15 and 25 repeated queries, well above the single-observation standard common in existing evaluations.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.HC).
Categorie arXiv: Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2609.33743 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:13:52.231Z dalla pagina https://arxiv.org/catchup/cs.HC/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).