Metraipolis › archive › dc_25f0712973b6

Multilingual in Name Only? Cultural and Linguistic Weaknesses of LLMs in Urdu — arXiv:2609.10758

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.10758
License (as archived)
CC0-1.0
Archived
2026-10-16T05:24:40.004Z
SHA-256 of the archived text
eaa810dea2b336afbd6c397880fd996272b6a326b0bace77f235715e3bdc1e3d
Stable URL of this page
https://metraipolis.com/documento/dc_25f0712973b6

How to cite

Multilingual in Name Only? Cultural and Linguistic Weaknesses of LLMs in Urdu — arXiv:2609.10758 — arXiv (en), https://arxiv.org/abs/2609.10758 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_25f0712973b6 (Metraipolis archive dc_25f0712973b6, sha256 eaa810dea2b336afbd6c397880fd996272b6a326b0bace77f235715e3bdc1e3d)
BibTeX
@misc{dc25f0712973b6,
  title = {Multilingual in Name Only? Cultural and Linguistic Weaknesses of LLMs in Urdu — arXiv:2609.10758},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.10758},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_25f0712973b6; sha256 eaa810dea2b336afbd6c397880fd996272b6a326b0bace77f235715e3bdc1e3d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.10758. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multilingual large language models (LLMs) are increasingly used for open-ended text generation, yet their behaviour in low-resource languages remains poorly understood. In this work, we question how correct and reliable is the generation of multilingual LLMs when used for the task of story generation. We consider Urdu language as a representative low-resource language. We generate Urdu-Stories, a corpus of 93 stories generated using three contemporary LLMs (GPT-5.1, Qwen-3-Max, DeepSeek-3.1). We manually annotate the errors present in them under a nine-label linguistic, semantic, and cultural taxonomy. Our notable findings suggest that LLMs often make basic errors of grammar and semantics. The stories lack coherence, have unnatural repetition and show pervasive cultural shallowness. We further show using few-shot prompting that the cultural and context errors largely remain unresolved. Our findings highlight the limitations of current LLMs as a reliable source of content generation and information retrieval for low-resource languages.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-11 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.10758 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T05:17:52.082Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).