Metraipolis › archive › dc_cc31cf2b689b
Swap, Replicate, Read the Mass: A Protocol for Auditing Demographic Bias in System One Decision Models — arXiv:2611.00496
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00496
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:30:50.968Z
- SHA-256 of the archived text
07e93e4a69746c41341439ed6eeb84ea0be260eb84246176333c997da3bf7d35- Stable URL of this page
https://metraipolis.com/documento/dc_cc31cf2b689b
How to cite
Swap, Replicate, Read the Mass: A Protocol for Auditing Demographic Bias in System One Decision Models — arXiv:2611.00496 — arXiv (en), https://arxiv.org/abs/2611.00496 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cc31cf2b689b (Metraipolis archive dc_cc31cf2b689b, sha256 07e93e4a69746c41341439ed6eeb84ea0be260eb84246176333c997da3bf7d35)
BibTeX
@misc{dccc31cf2b689b,
title = {Swap, Replicate, Read the Mass: A Protocol for Auditing Demographic Bias in System One Decision Models — arXiv:2611.00496},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00496},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cc31cf2b689b; sha256 07e93e4a69746c41341439ed6eeb84ea0be260eb84246176333c997da3bf7d35},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00496. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
System One decision models replace text generation with typed, probability-scored answers and are entering production. Answers execute without a person in the loop, and no multi-model audit exists on established bias benchmarks. We provide that audit: three benchmarks: BBQ (all eleven categories), CrowS-Pairs, and SilverSpoon, adapted to the typed interface, twelve models plus a no-thinking ablation, and 1,638,000 decisions. Every order-sensitive item runs twice with options swapped; native typed models also swap labels; ten configurations repeat under extra seeds and batch 16. We find widespread stereotype alignment, order-dependent answers, and confidence that tracks instability rather than bias. In BBQ, 97 of 143 model-by-category scores are significant and 94 aligned, while the commercial reference abstains on 99.5% of ambiguous items, hiding aligned probability mass. Option swaps flip 0.0% to 72.6% of choices; typed scoring stays below 10% except Laya (22-33%), and untuned letter prompting reaches 73%. CrowS-Pairs preference spans 0.496 to 0.816, and boolean answers can be decided by position (1.00 versus 0.005). SilverSpoon judgments shift toward lower-SES labels in five of eight decision models, inherited from the 9B backbone or induced by tuning. Audits must permute options and reverse questions by default; order-invariant scoring should become a procurement requirement. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.00496 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:29:48.450Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).