Metraipolis › archive › dc_84eb682a4f02
Actions Speak Louder than Words: Measuring Cross-Lingual Policy Retention in Tool-Using Agents — arXiv:2608.11110
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.11110
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T06:14:24.862Z
- SHA-256 of the archived text
9146db00a394f5a63b487c513cf4f31408b255c366818bceaef72ff6fe9bbe77- Stable URL of this page
https://metraipolis.com/documento/dc_84eb682a4f02
How to cite
Actions Speak Louder than Words: Measuring Cross-Lingual Policy Retention in Tool-Using Agents — arXiv:2608.11110 — arXiv (en), https://arxiv.org/abs/2608.11110 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_84eb682a4f02 (Metraipolis archive dc_84eb682a4f02, sha256 9146db00a394f5a63b487c513cf4f31408b255c366818bceaef72ff6fe9bbe77)
BibTeX
@misc{dc84eb682a4f02,
title = {Actions Speak Louder than Words: Measuring Cross-Lingual Policy Retention in Tool-Using Agents — arXiv:2608.11110},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.11110},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_84eb682a4f02; sha256 9146db00a394f5a63b487c513cf4f31408b255c366818bceaef72ff6fe9bbe77},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11110. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
When a tool-using agent is given the same task in a different language, does it still take the same steps? Multilingual evaluation rarely asks: it compares final answers and discards the actions. Yet those actions are the product: they fix cost and latency, decide how the system fails, and are the only auditable part of its behaviour. We make the action policy the measured object across 8 models, 6 parallel benchmarks and 41 languages (2.38M rollouts). The naive measurement fails: five confounds sit between raw trace similarity and any defensible claim, each able to flip a conclusion. Short traces score higher, empty traces score perfectly, unrelated traces agree by chance over half the time, the gap is capped by each model's reproducibility, and a model asked the same question twice in one language answers differently, leaving no baseline. We remove all five, and every correction makes the effect larger. Divergence proves structural, not sampling noise: it survives greedy decoding in every cell and stays flat as temperature rises, even as models grow less self-consistent. Normalised by their own reproducibility, four very different frontier models converge under greedy decoding, each keeping 71-73% of its action policy across languages, with model identity explaining only 5.7% of the variance. Below roughly 10B parameters it breaks down, and the ordering among smaller models is largely an artifact of a chance floor we measure by permutation rather than assume. Agents route non-English tasks through English; this pivot is causally load-bearing, confirmed by a pre-registered prediction across four models, and models will not abandon it when told to. Finally, a single trace-extraction regex, not the model, manufactured a multilingual failure: two worked examples raise one model's measured accuracy twenty-sixfold while its accuracy on readable outputs barely moves. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-12 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.11110 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T06:08:50.344Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).