Metraipolis › archive › dc_a744ad810792
BabelArena: A Large-Scale Multilingual Benchmark for LLM Agents — arXiv:2609.23490
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.23490
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:49:50.924Z
- SHA-256 of the archived text
099d85ea0a5e9adc619aa5de9255a906f16c7cc53b515bfa5c7c2fba01662a95- Stable URL of this page
https://metraipolis.com/documento/dc_a744ad810792
How to cite
BabelArena: A Large-Scale Multilingual Benchmark for LLM Agents — arXiv:2609.23490 — arXiv (en), https://arxiv.org/abs/2609.23490 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a744ad810792 (Metraipolis archive dc_a744ad810792, sha256 099d85ea0a5e9adc619aa5de9255a906f16c7cc53b515bfa5c7c2fba01662a95)
BibTeX
@misc{dca744ad810792,
title = {BabelArena: A Large-Scale Multilingual Benchmark for LLM Agents — arXiv:2609.23490},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.23490},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a744ad810792; sha256 099d85ea0a5e9adc619aa5de9255a906f16c7cc53b515bfa5c7c2fba01662a95},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.23490. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language model (LLM) agents increasingly execute multi-step workflows through tool use and interaction with users and environments. However, current agent evaluations are largely English-centric, limiting our understanding of agent capabilities in multilingual settings. We introduce BabelFlow, a benchmark-general agentic workflow that adapts existing agent benchmarks to new languages by analyzing runtime dependencies, coordinating structure-preserving translation, and combining multi-layer verification with human review to preserve task and evaluation semantics. Using BabelFlow, we construct BabelArena, a task-aligned benchmark comprising 16,146 instances derived from 702 canonical tasks across four benchmark families, 13 domains, and 23 languages. Experiments with five frontier models show that no single model dominates across benchmark families and that cross-language disparities extend well beyond task success. Lower-resource languages exhibit distinct failure patterns, with larger shares of tool-use and control-flow errors rather than answer-quality errors alone, pointing to gaps in reliable task execution across the resource levels of these languages. On the same tasks, agents in low-resource languages also consume substantially more tokens than in English (up to roughly twice the input) without proportional increases in interaction length, and language consistency degrades further on tasks requiring structured output, where switches are directed overwhelmingly toward English. We believe BabelArena provides a foundation for advancing research on reliable and efficient multilingual agents. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.23490 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T06:48:44.993Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).