Metraipolis › archive › dc_7ee3d3b00496

Evaluating Cultural Awareness of LLMs for Haitian Creole — arXiv:2609.31506

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.31506
License (as archived)
CC0-1.0
Archived
2026-10-08T01:39:59.270Z
SHA-256 of the archived text
a6bed4baed5917070ae9d18170556759140b49efad6fb33c565c9e350acb081c
Stable URL of this page
https://metraipolis.com/documento/dc_7ee3d3b00496

How to cite

Evaluating Cultural Awareness of LLMs for Haitian Creole — arXiv:2609.31506 — arXiv (en), https://arxiv.org/abs/2609.31506 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7ee3d3b00496 (Metraipolis archive dc_7ee3d3b00496, sha256 a6bed4baed5917070ae9d18170556759140b49efad6fb33c565c9e350acb081c)
BibTeX
@misc{dc7ee3d3b00496,
  title = {Evaluating Cultural Awareness of LLMs for Haitian Creole — arXiv:2609.31506},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31506},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7ee3d3b00496; sha256 a6bed4baed5917070ae9d18170556759140b49efad6fb33c565c9e350acb081c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31506. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) exhibit substantial performance disparities between high- and low-resource languages. Beyond lower task performance, they often fail to capture the cultural norms and values of underrepresented communities. In this work, we present the first systematic evaluation of cultural awareness in LLMs for Haitian Creole, a language spoken by millions but severely underrepresented in digital resources. We assess cultural awareness along four complementary dimensions---specificity, bias, diversity, and variation---using a benchmark of culturally salient prompts curated by native speakers in a text infilling setting. Our results reveal a clear gap between cultural awareness in Haitian Creole and higher-resource French, with Haitian performance being more uneven across domains and more affected by French linguistic interference. Story generation further reveals recurring portrayals of Haitian characters through hardship and resilience, showing that even positive characterizations can encode stereotypical narratives. Our code, benchmark, and evaluation framework are publicly available.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.31506 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:12:14.608Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).