Metraipolis › archive › dc_ca97cdb07602

CuBEs: Culturally-Situated Behavioral Evaluations and the Limitations of Culture-Blind LLM Judges — arXiv:2610.02622

Source: arXiv
Original: https://arxiv.org/abs/2610.02622
License (as archived): CC0-1.0
Archived: 2026-10-05T22:49:06.615Z
SHA-256 of the archived text: bb4847d7a3ece1961cbfa1c5185fe1011e991f66ff10e006a913020415215dc1
Stable URL of this page: https://metraipolis.com/documento/dc_ca97cdb07602

How to cite

CuBEs: Culturally-Situated Behavioral Evaluations and the Limitations of Culture-Blind LLM Judges — arXiv:2610.02622 — arXiv, https://arxiv.org/abs/2610.02622 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ca97cdb07602 (Metraipolis archive dc_ca97cdb07602, sha256 bb4847d7a3ece1961cbfa1c5185fe1011e991f66ff10e006a913020415215dc1)

BibTeX

@misc{dcca97cdb07602,
  title = {CuBEs: Culturally-Situated Behavioral Evaluations and the Limitations of Culture-Blind LLM Judges — arXiv:2610.02622},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.02622},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ca97cdb07602; sha256 bb4847d7a3ece1961cbfa1c5185fe1011e991f66ff10e006a913020415215dc1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02622. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Evaluating the occurrence and triggers of large language model (LLM) behaviors - such as sycophancy, self-preference, or over-confidence - is critical for predicting real-world model deployment risks. However, existing situated behavioral evaluations typically ignore cultural context, limiting their generalizability across an increasingly global user base. To address this gap, we propose CuBEs - Culturally-situated Behavior Evaluations that probe for response patterns across diverse user cultures. We first extend an automated testing pipeline to inject cultural context into behavioral test scenarios and subsequent evaluation. We assess the cultural adaptability of this pipeline by building a human-labeled dataset that captures nuanced dimensions of behavior understanding across 12 distinct cultures. Our dataset reveals significant cross-cultural variations that one-size-fits all judgments fail to capture. Through evaluating 13 open- and closed-source LLMs, we find that introducing cultural situatedness in the evaluation scenario creates significant variation in the presence of a behavior. For example, while our baseline experiments testing for political bias capture localized Western political dimensions like the American conservative-progressive divide, non-Western culturally situated evaluations surface entirely different axes of bias such as religious and colonial political issues. Our findings demonstrate that standard, culturally-agnostic evaluations fail to capture these shifts, highlighting the necessity of culturally situated behavioral testing for global deployments.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.02622 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T22:45:55.609Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).