Metraipolis › archive › dc_77547c500ff4

Red-Teaming Large Language Models in Low-Resource African Languages: A Case Study of Yoruba and Nigerian Pidgin — arXiv:2610.39590

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.39590
License (as archived)
CC0-1.0
Archived
2026-10-31T06:13:20.203Z
SHA-256 of the archived text
76dfc5ae8a391235c1b4f0398955336a911cb5b124a7c96fa7a2842e42ea53fc
Stable URL of this page
https://metraipolis.com/documento/dc_77547c500ff4

How to cite

Red-Teaming Large Language Models in Low-Resource African Languages: A Case Study of Yoruba and Nigerian Pidgin — arXiv:2610.39590 — arXiv (en), https://arxiv.org/abs/2610.39590 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_77547c500ff4 (Metraipolis archive dc_77547c500ff4, sha256 76dfc5ae8a391235c1b4f0398955336a911cb5b124a7c96fa7a2842e42ea53fc)
BibTeX
@misc{dc77547c500ff4,
  title = {Red-Teaming Large Language Models in Low-Resource African Languages: A Case Study of Yoruba and Nigerian Pidgin — arXiv:2610.39590},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.39590},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_77547c500ff4; sha256 76dfc5ae8a391235c1b4f0398955336a911cb5b124a7c96fa7a2842e42ea53fc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.39590. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As large language models (LLMs) become more widespread in Africa, their safety in low-resource languages remains largely untested since most red-teaming work focuses on English. This study evaluates six state-of-the-art LLMs using 400 explicit adversarial prompts in Yoruba and Nigerian Pidgin, categorised into four harm categories. The responses were scored by native-speaker annotators and reliability was assessed using Cohen's $\kappa$. While models that rejected up to 69% of English prompts rejected as few as 0% in Yoruba, this was not because they were safer, but because they failed to understand the prompts. Attack success rates increased for most models in the Nigerian languages, with GPT-4o Mini demonstrating the most significant cross-lingual safety discrepancy (+0.23 in Yoruba). When models did engage with adversarial prompts, many of the responses were harmful, with rates of harmful content reaching 0.63 in English and 0.50 in Nigerian Pidgin. The most difficult harm category to detect was cultural stereotypes and discrimination (ASR = 0.44 in Nigerian Pidgin). Comprehension in Yoruba was low across most models, and this directly suppressed apparent attack success without indicating actual safety. Gemini 2.0 Flash was the most consistent model across languages. Despite its explicit training in African languages, Aya 101 had the highest attack success rate in English (0.60) and one of the lowest Yoruba comprehension rates (0.28). These results demonstrate that English-centric safety alignment does not transfer to Nigerian languages, highlighting the urgent need for language-specific safety evaluations.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.39590 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T06:07:08.725Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).