Metraipolis › archive › dc_daa1798dd30e

Evaluating the Robustness of Japanese LLMs to IME-Related and Typographical Errors — arXiv:2610.01241

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.01241
License (as archived)
CC0-1.0
Archived
2026-10-06T13:55:17.141Z
SHA-256 of the archived text
3ca6d6f29d3d25a50780392d027b958a6f88cb046c39e22ecc681eb6c33406ee
Stable URL of this page
https://metraipolis.com/documento/dc_daa1798dd30e

How to cite

Evaluating the Robustness of Japanese LLMs to IME-Related and Typographical Errors — arXiv:2610.01241 — arXiv (en), https://arxiv.org/abs/2610.01241 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_daa1798dd30e (Metraipolis archive dc_daa1798dd30e, sha256 3ca6d6f29d3d25a50780392d027b958a6f88cb046c39e22ecc681eb6c33406ee)
BibTeX
@misc{dcdaa1798dd30e,
  title = {Evaluating the Robustness of Japanese LLMs to IME-Related and Typographical Errors — arXiv:2610.01241},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.01241},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_daa1798dd30e; sha256 3ca6d6f29d3d25a50780392d027b958a6f88cb046c39e22ecc681eb6c33406ee},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01241. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) have achieved strong performance across various natural language processing tasks. However, their robustness to typographical errors remains underexplored, particularly in Japanese, where text input involves multiple writing systems and IME-based conversion. In this study, we evaluate the robustness of Japanese LLMs against realistic Japanese-specific typos. We introduce five typo categories: Character Transposition, Character Replacement, Homophone Conversion, Japanese IME Conversion, and Full-Width Conversion. These perturbations are applied to three Japanese benchmark datasets (JMMLU, JCommonsenseQA, and JamC-QA), and eleven Japanese and multilingual LLMs are evaluated. The results show that Character Transposition and Character Replacement typos consistently reduce accuracy across benchmarks, whereas IME Conversion, Full-Width Conversion, and Homophone Conversion have relatively limited impact. These findings reveal that current Japanese LLMs remain vulnerable to realistic Japanese typing errors, particularly those that substantially distort the original input, highlighting the importance of robustness evaluation in practical input environments.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.01241 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T12:15:34.699Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).