Metraipolis › archive › dc_63ce04f20835
WarBias: A Bilingual LLM Benchmark of Stereotypes About Internally Displaced People and Veterans — arXiv:2611.11284
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.11284
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-10T18:44:37.003Z
- SHA-256 of the archived text
66d4085d1a02a1f6a4b146c3b0c215264e2df8d435728272452359263c3e54be- Stable URL of this page
https://metraipolis.com/documento/dc_63ce04f20835
How to cite
WarBias: A Bilingual LLM Benchmark of Stereotypes About Internally Displaced People and Veterans — arXiv:2611.11284 — arXiv (en), https://arxiv.org/abs/2611.11284 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_63ce04f20835 (Metraipolis archive dc_63ce04f20835, sha256 66d4085d1a02a1f6a4b146c3b0c215264e2df8d435728272452359263c3e54be)
BibTeX
@misc{dc63ce04f20835,
title = {WarBias: A Bilingual LLM Benchmark of Stereotypes About Internally Displaced People and Veterans — arXiv:2611.11284},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.11284},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_63ce04f20835; sha256 66d4085d1a02a1f6a4b146c3b0c215264e2df8d435728272452359263c3e54be},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.11284. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
This paper contains examples of stereotypes about veterans and displaced people that may be offensive or distressing. Social bias benchmarks for language models commonly examine categories such as gender, race, and religion, often drawn from lists of legally protected characteristics. Veterans and internally displaced people (IDPs) fall outside these lists, and stereotypes about them, which can undermine access to employment, housing, and social participation, remain largely absent from language model evaluation. We introduce WarBias, a Ukrainian--English benchmark of stereotypes about these two groups. Built from a taxonomy of 120 stereotype claims grounded in research and monitoring reports, WarBias contains 1,007 stereotype--counter-stereotype--unrelated sentence triplets and 240 multiple-choice questions per language. A Ukrainian language model selected in a blinded human pilot generated the Ukrainian triplets. At least two independent annotators reviewed each triplet in the annotation cohort. The authors revised items in response to the reviews, and the final release was machine-translated into English. Each claim carries a human severity rating. In a baseline evaluation of six models, four prefer stereotypical Ukrainian sentences about at least one group, most strongly about IDPs, while English scores are mostly balanced. WarBias supports complementary evaluations of stereotypical preferences and context-sensitive judgments, and provides paired forget/retain material for unlearning-based bias mitigation. We document the benchmark's construction, intended uses, and limitations, and offer its construction method as a template for analogous benchmarks in other conflict-affected societies. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.11284 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-10T18:37:52.437Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).