Metraipolis › archive › dc_6ab9f4b0d87f
Quantitative Measurement of Language Distance among Closely Related Indo-European Languages Using Pretrained Language Models: A Case Study on the North Germanic Branch — arXiv:2609.34152
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.34152
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T21:53:19.841Z
- SHA-256 of the archived text
ab794f5848b31116d5aab17300f9a4a3760b0d79fc7b4919a007949b486b6710- Stable URL of this page
https://metraipolis.com/documento/dc_6ab9f4b0d87f
How to cite
Quantitative Measurement of Language Distance among Closely Related Indo-European Languages Using Pretrained Language Models: A Case Study on the North Germanic Branch — arXiv:2609.34152 — arXiv (en), https://arxiv.org/abs/2609.34152 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6ab9f4b0d87f (Metraipolis archive dc_6ab9f4b0d87f, sha256 ab794f5848b31116d5aab17300f9a4a3760b0d79fc7b4919a007949b486b6710)
BibTeX
@misc{dc6ab9f4b0d87f,
title = {Quantitative Measurement of Language Distance among Closely Related Indo-European Languages Using Pretrained Language Models: A Case Study on the North Germanic Branch — arXiv:2609.34152},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.34152},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6ab9f4b0d87f; sha256 ab794f5848b31116d5aab17300f9a4a3760b0d79fc7b4919a007949b486b6710},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34152. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Among closely related North Germanic languages, the quantification of language distance has traditionally relied on qualitative methods, lacking a unified multi-dimensional computational framework. Multilingual pretrained models based on the Transformer architecture can map texts from different languages into a shared vector space, enabling quantitative measurement of language distance. This paper focuses on the three North Germanic languages---Danish, Norwegian (Bokmål), and Swedish---and proposes a three-metric quantitative framework based on pretrained language models: (1)~sentence-level semantic distance, computed as cosine similarity between LaBSE and mBERT encodings of parallel sentences; (2)~orthographic fragmentation rate, measuring subword tokenization efficiency when cross-applying monolingual BERT vocabularies to parallel texts; (3)~MLM predictability, comparing prediction confidence and entropy in masked language modeling using mBERT across languages. Using 150 trilingual parallel sentence triplets from the Tatoeba corpus as controlled samples, we obtain consistent distance rankings on two independent models: LaBSE: da--no $0.012 < $ no--sv $0.016 < $ da--sv $0.020$; mBERT: da--no $0.016 < $ no--sv $0.045 \approx $ da--sv $0.046$. This ranking is consistent with the historical linguistic conclusion that ``400 years of Danish rule over Norway (1380--1814) led to highly cognate written languages.'' The three metrics---semantic, orthographic, and predictability---converge on the same conclusion, providing a reproducible computational framework for the quantitative study of distance among closely related languages, extensible in principle to more branches of the Indo-European language family, pending validation on additional language groups. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.34152 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T21:27:08.589Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).