Metraipolis › archive › dc_ea06668a4fc2

Left-Branching Transformers Excel at Right-Branching Languages: Data Shapes Word Order Preferences in Language Models — arXiv:2608.15129

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.15129
License (as archived)
CC0-1.0
Archived
2026-10-18T18:00:29.869Z
SHA-256 of the archived text
fcd82e5bc37b79f7a2f6e8c8cc45d93538bcf8cfeecbe9735a5ee409cc837b2f
Stable URL of this page
https://metraipolis.com/documento/dc_ea06668a4fc2

How to cite

Left-Branching Transformers Excel at Right-Branching Languages: Data Shapes Word Order Preferences in Language Models — arXiv:2608.15129 — arXiv (en), https://arxiv.org/abs/2608.15129 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ea06668a4fc2 (Metraipolis archive dc_ea06668a4fc2, sha256 fcd82e5bc37b79f7a2f6e8c8cc45d93538bcf8cfeecbe9735a5ee409cc837b2f)
BibTeX
@misc{dcea06668a4fc2,
  title = {Left-Branching Transformers Excel at Right-Branching Languages: Data Shapes Word Order Preferences in Language Models — arXiv:2608.15129},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.15129},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ea06668a4fc2; sha256 fcd82e5bc37b79f7a2f6e8c8cc45d93538bcf8cfeecbe9735a5ee409cc837b2f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15129. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We systematically compare word order preferences in decoder-only language models across 192 artificial languages and typologically diverse natural languages. On artificial languages, models exhibit a left-branching preference that aligns with neither natural language universals nor human word order learning biases. On natural languages, monolingual models show no clear base word order bias at small scales, but as data grows, a preference for right-branching subject-verb-object (SVO) languages emerges while SOV falls behind despite being the most frequent order cross-linguistically. This SVO advantage extends to multilingual models and correlates with language resource level and data quality rather than word order. Thus, the same architecture exhibits opposite preferences on artificial and natural languages, establishing that word order biases observed in practice are data-driven. Since highly-resourced languages are overwhelmingly SVO, these biases risk gradually reducing word order diversity, particularly in languages that productively use multiple word orders, with the widespread adoption of LLMs.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.15129 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T16:29:13.989Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).