Metraipolis › archive › dc_06e6c97cf7bc
On the Framing Effect of AI Red-Teaming Instructions: Evidence from Prior Exercises and Controlled Simulations — arXiv:2611.00412
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00412
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:25:48.586Z
- SHA-256 of the archived text
849d7cb54c2a80ea63e45eddc7228a47f3f32ed4e1a9ecb62bb54c506e8aba79- Stable URL of this page
https://metraipolis.com/documento/dc_06e6c97cf7bc
How to cite
On the Framing Effect of AI Red-Teaming Instructions: Evidence from Prior Exercises and Controlled Simulations — arXiv:2611.00412 — arXiv (en), https://arxiv.org/abs/2611.00412 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_06e6c97cf7bc (Metraipolis archive dc_06e6c97cf7bc, sha256 849d7cb54c2a80ea63e45eddc7228a47f3f32ed4e1a9ecb62bb54c506e8aba79)
BibTeX
@misc{dc06e6c97cf7bc,
title = {On the Framing Effect of AI Red-Teaming Instructions: Evidence from Prior Exercises and Controlled Simulations — arXiv:2611.00412},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00412},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_06e6c97cf7bc; sha256 849d7cb54c2a80ea63e45eddc7228a47f3f32ed4e1a9ecb62bb54c506e8aba79},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00412. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
AI red-teaming, adversarially probing generative AI systems to surface harms, has become a central evaluation method. However, prior work has highlighted that red-teaming is a fairly open-ended task, laden with myriad design choices that can shape its findings. In this work, we focus on the instructions used to conduct red-teaming exercises and hypothesize that they can significantly affect outcomes in a manner analogous to the framing effect in psychology, where the format in which information is presented greatly impacts how people respond. To understand whether and why instructions impact red-teaming outcomes, first, we analyze the range of ways in which instructions have historically been formulated, reviewing 228 instances of prior work. Second, based on takeaways from this review, we run controlled simulations that hold parameters such as the risk domain, the attacker, and the model under investigation fixed while varying only the instructions. From five real instruction templates used by industry and academic organizations, we derive 45 instruction frames across three risk domains and three levels of risk granularity, generating 100 variants per frame, and deploy each against five open-weights target models, for 22,500 attacks. Using a generalized linear mixed model, we find that certain instruction properties significantly predict attack success rates: the lowest and highest levels of risk granularity both outperform intermediate specificity, as do short and long instructions relative to medium-length ones. The presence of examples is associated with higher attack success rates, while supplying explicit harm definitions yields lower ones. As alignment evaluations become increasingly automated, our results suggest that humans continue to shape red-teaming outcomes substantially through the instructions they provide to scaffold the activity. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computers and Society (cs.CY) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.00412 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:19:50.002Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).