Metraipolis › archive › dc_2e8458c39bb9
Multilingual GSM-Symbolic: What determines capability transfer across languages? — arXiv:2610.03367
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.03367
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-05T22:49:38.120Z
- SHA-256 of the archived text
b0b674a25ff02e45bd29321912211d0c295be9d8ca44c16ae4ce36934a0a00f1- Stable URL of this page
https://metraipolis.com/documento/dc_2e8458c39bb9
How to cite
Multilingual GSM-Symbolic: What determines capability transfer across languages? — arXiv:2610.03367 — arXiv, https://arxiv.org/abs/2610.03367 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2e8458c39bb9 (Metraipolis archive dc_2e8458c39bb9, sha256 b0b674a25ff02e45bd29321912211d0c295be9d8ca44c16ae4ce36934a0a00f1)
BibTeX
@misc{dc2e8458c39bb9,
title = {Multilingual GSM-Symbolic: What determines capability transfer across languages? — arXiv:2610.03367},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.03367},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2e8458c39bb9; sha256 b0b674a25ff02e45bd29321912211d0c295be9d8ca44c16ae4ce36934a0a00f1},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.03367. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We understand little about how capabilities acquired in one language carry over to another, or what governs this transfer: evaluations rely on incomparable, saturation-prone datasets and rarely examine its determinants jointly. Identifying what predicts transfer would let us avoid exhaustive evaluation across all language pairs and let developers target the factors that limit performance in low-resource languages. To evaluate cross-lingual capability transfer, we introduce Multilingual GSM-Symbolic, an extensible multilingual mathematical dataset covering 30,000 item-matched question-answer pairs and spanning 15 languages. It utilises symbolic templates to prevent overfitting and ensure generalisation by allowing generation of millions of high-quality variations from a single sample. Using Multilingual GSM-Symbolic, we quantify the largest determinants of capability as model size ($\beta = 1.77$), language resource level ($\beta = 0.77$), reasoning ($\beta = 0.67$) and typological distance ($\beta = -0.25$). This joint estimation allows these determinants to be expressed in terms of one another: a 32B model evaluated in Marathi performs like a 10B model in English. Our findings have important implications for model developers, showing that model size and reasoning narrow the performance gap between low- and high-resource languages ($\beta = -0.27$ and $\beta = -0.20$, respectively), while similar levers have little or no effect on typologically distant languages. Overall, our analysis framework explains 92% of between-language variation, but only 23% of the model-by-language variation, and predicts a model's performance on an unseen language within 6.0pp (r=.96). Incorporating measurements from just 10 templates in the target language reduces this to 4.19pp, enabling reasonable estimates of performance with little or no downstream dataset. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.03367 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-05T22:45:55.695Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).