Metraipolis › archive › dc_59d21f85576b

Double Trouble: Bilingual Pretraining Leaves Language-Conditioned Effects in Shared-Language Representations — arXiv:2608.26576

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.26576
License (as archived)
CC0-1.0
Archived
2026-10-17T15:19:35.698Z
SHA-256 of the archived text
33559a888fe2e75886d5a524ff77c99d60cde60b0784b7f3586a60b8e79a086d
Stable URL of this page
https://metraipolis.com/documento/dc_59d21f85576b

How to cite

Double Trouble: Bilingual Pretraining Leaves Language-Conditioned Effects in Shared-Language Representations — arXiv:2608.26576 — arXiv (en), https://arxiv.org/abs/2608.26576 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_59d21f85576b (Metraipolis archive dc_59d21f85576b, sha256 33559a888fe2e75886d5a524ff77c99d60cde60b0784b7f3586a60b8e79a086d)
BibTeX
@misc{dc59d21f85576b,
  title = {Double Trouble: Bilingual Pretraining Leaves Language-Conditioned Effects in Shared-Language Representations — arXiv:2608.26576},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.26576},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_59d21f85576b; sha256 33559a888fe2e75886d5a524ff77c99d60cde60b0784b7f3586a60b8e79a086d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.26576. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A concept can carry different associations across languages, while modern language models learn English alongside many other languages during pretraining. Yet comparisons among existing models cannot easily isolate how any one language changes the way these models represent English concepts because their training corpora, compute, architectures, and random seeds all differ. We study this question through a controlled experiment with 40 matched 310M-parameter decoder-only models that share an architecture, tokenizer, training recipe, and English data source. Each bilingual condition adds one of eight languages, while four experimental comparisons separately account for English exposure, total training, and English-document overlap. We align each model pair using 3,000 common English words, then measure where 1,000 held-out English concepts fall along 50 fixed semantic contrasts, such as red versus white. Across 32 experimental comparisons, English concept positions differ more between bilingual and English-only conditions than between English-only runs with different random seeds. These differences are larger in contextual states than in token embeddings and peak in middle layers. The language learned alongside English can therefore change how a model represents English concepts even when its English input representations are explicitly aligned.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.26576 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T15:15:18.060Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).