Metraipolis › archive › dc_df03a8203329

Benchmarking Arabic Dialects for Math Problems — arXiv:2610.28607

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.28607
License (as archived)
CC0-1.0
Archived
2026-10-21T07:08:38.915Z
SHA-256 of the archived text
a62c15d78042cc496c4b848957b8dac8f1cc413783729010595a530e6a3717fb
Stable URL of this page
https://metraipolis.com/documento/dc_df03a8203329

How to cite

Benchmarking Arabic Dialects for Math Problems — arXiv:2610.28607 — arXiv (en), https://arxiv.org/abs/2610.28607 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_df03a8203329 (Metraipolis archive dc_df03a8203329, sha256 a62c15d78042cc496c4b848957b8dac8f1cc413783729010595a530e6a3717fb)
BibTeX
@misc{dcdf03a8203329,
  title = {Benchmarking Arabic Dialects for Math Problems — arXiv:2610.28607},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.28607},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_df03a8203329; sha256 a62c15d78042cc496c4b848957b8dac8f1cc413783729010595a530e6a3717fb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.28607. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Mathematical reasoning remains underexplored for low-resource languages, including Dialectal Arabic. To bridge this gap, we introduce the DAR-MGSM (Dialectal Arabic Multilingual Grade School Math) Benchmark, the first manually curated benchmark for evaluating mathematical reasoning in Dialectal Arabic. DAR-MGSM consists of math word problems (MWPs) requiring multi-step reasoning and covers five major Arabic dialects alongside Modern Standard Arabic (MSA). The benchmark comprises two parallel versions: translated-only and translated & culturally localized. The latter version adapts entities and references to regional varieties. We evaluate eleven Arabic-centric and multilingual large language models on DAR-MGSM. The results show that the performance generally declines on the localized version, highlighting the challenges introduced by culturally grounded adaptations. However, the magnitude of this effect varies across models, with Arabic-centric models exhibiting different levels of sensitivity to localization. A mechanistic interpretability analysis shows that translated and localized variants of the same MWPs often induce distinct internal reasoning trajectories, even when aggregate accuracy remains similar. Overall, our results show that mathematical reasoning performance depends on both linguistic variety and cultural context.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.28607 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:43.673Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).