Metraipolis › archive › dc_43043db84ade
How Robust Are LLMs to Vietnamese Dialects? — arXiv:2608.10414
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.10414
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T06:14:11.895Z
- SHA-256 of the archived text
588cf9e6d789ca8f74a53d067b0a4a14e584f0ff603b9a63d6ba3fbb7dec79e8- Stable URL of this page
https://metraipolis.com/documento/dc_43043db84ade
How to cite
How Robust Are LLMs to Vietnamese Dialects? — arXiv:2608.10414 — arXiv (en), https://arxiv.org/abs/2608.10414 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_43043db84ade (Metraipolis archive dc_43043db84ade, sha256 588cf9e6d789ca8f74a53d067b0a4a14e584f0ff603b9a63d6ba3fbb7dec79e8)
BibTeX
@misc{dc43043db84ade,
title = {How Robust Are LLMs to Vietnamese Dialects? — arXiv:2608.10414},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.10414},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_43043db84ade; sha256 588cf9e6d789ca8f74a53d067b0a4a14e584f0ff603b9a63d6ba3fbb7dec79e8},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.10414. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large Language Models (LLMs) are typically evaluated on standard written Vietnamese, yet everyday communication frequently involves regional dialects that preserve meaning but differ in surface form. Existing Vietnamese dialect work largely addresses this issue through dialect-to-standard normalization instead of measuring how the model fails under Vietnamese dialectal inputs. To address this gap, we present the first systematic evaluation of LLM robustness to Vietnamese dialect variation across multiple tasks, quantifying performance degradation and failure patterns. We introduce VialectBench (Vietnamese Dialects Benchmarking), a controlled benchmark for testing whether model decisions remain stable across six Vietnamese dialect groups. VialectBench contains 400 Standard Vietnamese source instances and 2,400 human-written dialectal rewrites spanning emotion recognition (ER), natural language inference (NLI), question answering (QA), and multiple-choice question answering (MCQA). Dataset evaluation with a fixed reference language model shows that the dialectal rewrites induce a measurable model-relative likelihood shift while remaining nearly equal in length to their Standard counterparts. Across ten instruction-tuned models, dialectal inputs reduce average performance by 2.82%, and no evaluated model is fully dialect-invariant. All four tasks are affected, with QA showing the largest average degradation. Robustness also varies substantially across dialect groups: PNT3 and PNT2 cause the largest average performance drops, at 6.17% and 4.73%, respectively, whereas PNB slightly improves average performance by 0.42%. The Central dialect group (PNT1-PNT4) also yields the highest average harmful-flip rate across all models, at 6.54%. These findings show that strong performance on Standard Vietnamese does not guarantee reliable behavior under meaning-preserving regional variation. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-12 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.10414 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T06:08:50.314Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).