Metraipolis › archive › dc_296e1004b44f
Language Non-Invariance in the Ideological Evaluation of Large Language Models — arXiv:2610.41135
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.41135
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T07:48:26.617Z
- SHA-256 of the archived text
69008557f6ccee0fa8ccaf1f2682a07ee7f2f73d7f1c8c7b34e1a0a75ebe4af8- Stable URL of this page
https://metraipolis.com/documento/dc_296e1004b44f
How to cite
Language Non-Invariance in the Ideological Evaluation of Large Language Models — arXiv:2610.41135 — arXiv (en), https://arxiv.org/abs/2610.41135 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_296e1004b44f (Metraipolis archive dc_296e1004b44f, sha256 69008557f6ccee0fa8ccaf1f2682a07ee7f2f73d7f1c8c7b34e1a0a75ebe4af8)
BibTeX
@misc{dc296e1004b44f,
title = {Language Non-Invariance in the Ideological Evaluation of Large Language Models — arXiv:2610.41135},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.41135},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_296e1004b44f; sha256 69008557f6ccee0fa8ccaf1f2682a07ee7f2f73d7f1c8c7b34e1a0a75ebe4af8},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41135. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Recent multilingual studies show that political evaluations of large language models (LLMs) can vary across languages. This paper examines how robust this non-invariance is to questionnaire translation, surface wording, stochastic model variation, and instrument choice, and how much it matters for cross-model comparison. Using the Political Compass, the analysis compares the positions of five LLMs, GPT-3.5, GPT-4o, GPT-5.2, Qwen-2.5, and Mistral Large, on its economic and social axes in English, Spanish, and French, with repeated stochastic draws. Additional exercises use independent Spanish and French translations, paraphrased versions in all three languages, and an alternative ideological instrument. Measured ideological positions are not language-invariant: relative to English, between four and five of the ten pairwise model rankings reverse in Spanish or French, depending on the language and axis. The broad cross-language pattern persists under independent translations, indicating that it is not solely an artifact of the official questionnaire versions. Yet the magnitude of particular shifts, and in some cases their sign, remains sensitive to wording and instrument choice. The evidence therefore supports a measurement interpretation: ideological audits recover language- and instrument-conditioned outcomes rather than a language-free political attribute. Because a single provider serves distinct linguistic constituencies through a common product, without users observing what other constituencies receive, this lack of comparability also bears on political accountability and cross-model comparisons. Tema degli esploratori: governance digitale e beni comuni (elenco arXiv cs.CY, nuovi lavori del giorno). Categorie arXiv: Computers and Society (cs.CY). Pagina del lavoro: https://arxiv.org/abs/2610.41135 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T07:40:52.170Z dalla pagina https://arxiv.org/list/cs.CY/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).