Metraipolis › archive › dc_2e6af309dd45

arXiv 2610.10592: Wieszcz-XIX: A 3.1-Billion-Word Corpus of Pre-1918 Polish and Temporally Bounded Language Models Trained From Scratch

Source
Metraipolis archive
Original
https://arxiv.org/abs/2610.10592
License (as archived)
CC-BY-4.0
Archived
2026-10-10T05:55:15.283Z
SHA-256 of the archived text
cc2bef51b830b2a4a34bb3bf1f4dc10226f2f5f70ebf3b4e498af681492eb835
Stable URL of this page
https://metraipolis.com/documento/dc_2e6af309dd45

How to cite

arXiv 2610.10592: Wieszcz-XIX: A 3.1-Billion-Word Corpus of Pre-1918 Polish and Temporally Bounded Language Models Trained From Scratch — Metraipolis archive, https://arxiv.org/abs/2610.10592 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_2e6af309dd45 (Metraipolis archive dc_2e6af309dd45, sha256 cc2bef51b830b2a4a34bb3bf1f4dc10226f2f5f70ebf3b4e498af681492eb835)
BibTeX
@misc{dc2e6af309dd45,
  title = {arXiv 2610.10592: Wieszcz-XIX: A 3.1-Billion-Word Corpus of Pre-1918 Polish and Temporally Bounded Language Models Trained From Scratch},
  howpublished = {Metraipolis archive},
  url = {https://arxiv.org/abs/2610.10592},
  note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_2e6af309dd45; sha256 cc2bef51b830b2a4a34bb3bf1f4dc10226f2f5f70ebf3b4e498af681492eb835},
  year = {2026}
}

Archived text

Reproduced under CC-BY-4.0. Attribution: Metraipolis archive, https://arxiv.org/abs/2610.10592. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Historical Polish is well documented as a language but annotated in machine-readable form only to about a million words for the period this paper covers; the rest sits behind optical character recognition of variable quality. We present Wieszcz-XIX, a corpus of 6.75 billion tokens (about 3.1 billion words) in 294,369 documents, most of them periodical issues, of Polish published from 1800 to 1918, assembled from Wolne Lektury and the Internet Archive by a pipeline that filters, deduplicates, audits for post-1918 leakage and splits at the document level. It is over three orders of magnitude larger than the annotated corpus of the same period, and we quantify its defects: recognition corruption against a false-positive floor, near-identical duplication, which is removed, and post-1918 leakage, which is excluded from the training corpus itself down to a known residue of 0.04 to 0.38% of its bytes, found in the transcribed source, so the published corpus is the trained one document for document. On a hand-corrected sample the character error rate is 0.68% where the text is legible, and 45% of the sampled passages cannot be corrected. On it we train a ladder of decoder-only models from 47M to 349M parameters from scratch, and measure their temporal boundedness. Against two modern Polish base models, one far larger, the 349M shows a crossover, as does the 107M against the comparator of its size: post-1918 vocabulary costs them about 3.1 bits per byte more than period vocabulary, a gap the comparators do not show, and period vocabulary costs them fewer bits than it costs the comparators. Shown period text, the models keep its spelling and the comparators only partly. Adding parameters gains about twice as much as a second pass over the data. We release the corpus, code and weights. Content warning: the models reproduce period prejudice, including antisemitic statements.

Articolo: «Wieszcz-XIX: A 3.1-Billion-Word Corpus of Pre-1918 Polish and Temporally Bounded Language Models Trained From Scratch» — Kocur, Szymon (2026-10-06), arXiv:2610.10592, categoria cs.DL (biblioteche digitali).
Fonte: https://arxiv.org/abs/2610.10592
Licenza dell'articolo indicata su arXiv: CC-BY-4.0 — https://creativecommons.org/licenses/by/4.0. Qui solo titolo, autori e abstract (non modificato), con attribuzione; il testo integrale resta su arXiv.
Consultato il 2026-10-10T05:55:00Z dalla pagina pubblica dell'articolo (robots.txt rispettato). Depositato dalla Gilda dei Bibliotecari. Abstract in lingua originale.