Metraipolis › archive › dc_e05128e04353

LLMs Get Smarter from Targeted Synthetic Multilingual Data — arXiv:2608.15964

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.15964
License (as archived)
CC0-1.0
Archived
2026-10-18T18:04:51.714Z
SHA-256 of the archived text
a0b7f1253dba019e4beb1e01f6b5e85c10203c623c77a74a60eb9a70f4ceaa70
Stable URL of this page
https://metraipolis.com/documento/dc_e05128e04353

How to cite

LLMs Get Smarter from Targeted Synthetic Multilingual Data — arXiv:2608.15964 — arXiv (en), https://arxiv.org/abs/2608.15964 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e05128e04353 (Metraipolis archive dc_e05128e04353, sha256 a0b7f1253dba019e4beb1e01f6b5e85c10203c623c77a74a60eb9a70f4ceaa70)
BibTeX
@misc{dce05128e04353,
  title = {LLMs Get Smarter from Targeted Synthetic Multilingual Data — arXiv:2608.15964},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.15964},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e05128e04353; sha256 a0b7f1253dba019e4beb1e01f6b5e85c10203c623c77a74a60eb9a70f4ceaa70},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15964. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Language-specific competency (LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt. In other words, a language model outputs different (and potentially incorrect) responses to the same semantic query when prompted in different languages. Prior work attributes this to an internal misalignment of semantic representation across languages. Currently, there are two main approaches to address LSC in the literature: (1) routing all queries through English, improving performance, but limiting language expressivity to English; or (2) training on language-balanced data, equalizing model performance across languages, but reducing overall performance. In this work, we take a data centric perspective and introduce HOTFIXR: Hardness Optimized Training data For Improving X-Lingual Reasoning. It is a data generation framework that uses models to probe and learn a student model's multilingual weaknesses, and generates data to mitigate them. HOTFIXR can generate multilingual synthetic training data that can improve multilingual performance. We evaluate on three in-distribution tasks, three out-of-distribution tasks, and four out-of-distribution languages. On average, HOTFIXR (1) improves in-distribution performance by 6.2%, (2) reduces catastrophic forgetting (induced by fine-tuning) on OOD tasks by 3.7%, and (3) on OOD languages by 7.1%. Overall, as many real-world applications requires multilingual LLMs, our work contributes to the efforts of making LLMs multilingually proficient. We will release code upon acceptance.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.15964 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T16:29:14.064Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).