Metraipolis › archive › dc_35eb4efbb188

Scaling Makes It Possible: How Large Models Master Impossible Languages — arXiv:2611.02015

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.02015
License (as archived): CC0-1.0
Archived: 2026-11-03T10:57:38.530Z
SHA-256 of the archived text: 067efc70fd2295461a00b7cff67cc9163e58a264524d0ed931a5916689a1121c
Stable URL of this page: https://metraipolis.com/documento/dc_35eb4efbb188

How to cite

Scaling Makes It Possible: How Large Models Master Impossible Languages — arXiv:2611.02015 — arXiv (en), https://arxiv.org/abs/2611.02015 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_35eb4efbb188 (Metraipolis archive dc_35eb4efbb188, sha256 067efc70fd2295461a00b7cff67cc9163e58a264524d0ed931a5916689a1121c)

BibTeX

@misc{dc35eb4efbb188,
  title = {Scaling Makes It Possible: How Large Models Master Impossible Languages — arXiv:2611.02015},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02015},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_35eb4efbb188; sha256 067efc70fd2295461a00b7cff67cc9163e58a264524d0ed931a5916689a1121c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02015. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

While language models (LMs) excel in natural language tasks, their ability to learn impossible languages--those that violate the structural, cognitive, or logical constraints that make human languages learnable and functional--remains debatable. This controversy is further fueled by a divide between linguistic theorists and empirical researchers working with small-scale LMs. To address this controversy, we introduce semantic coherence. This concept measures how effectively semantic information is expressed in language structures and examines the learning capabilities of LMs across multiple scales. By fine-tuning billion-parameter LMs on impossible language samples, we demonstrate that larger LMs achieve similar performance on impossible and possible languages, whereas million-parameter LMs exhibit significant degradation when trained on impossible languages. This performance gap widens as semantic complexity increases. We further show that scaling up the LM size fundamentally shifts the learning focus from grammatical rules to semantic understanding, with billion-scale LMs exhibiting a threefold reduction in sensitivity to low-level linguistic perturbations while maintaining stable performance under high-level semantic perturbations.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.02015 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:49:38.614Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).