Metraipolis › archive › dc_e1eea45fc52c
Does a Norwegian further-trained model give more outdated legal values than its base model? A preregistered paired comparison of Borealis 2 and Gemma 4 — arXiv:2611.02334
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.02334
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:10:47.508Z
- SHA-256 of the archived text
83e565f33aa8b3e06c2b4cfc777321dd5f791b347b6611ab86372727e3db9da2- Stable URL of this page
https://metraipolis.com/documento/dc_e1eea45fc52c
How to cite
Does a Norwegian further-trained model give more outdated legal values than its base model? A preregistered paired comparison of Borealis 2 and Gemma 4 — arXiv:2611.02334 — arXiv (en), https://arxiv.org/abs/2611.02334 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e1eea45fc52c (Metraipolis archive dc_e1eea45fc52c, sha256 83e565f33aa8b3e06c2b4cfc777321dd5f791b347b6611ab86372727e3db9da2)
BibTeX
@misc{dce1eea45fc52c,
title = {Does a Norwegian further-trained model give more outdated legal values than its base model? A preregistered paired comparison of Borealis 2 and Gemma 4 — arXiv:2611.02334},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.02334},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e1eea45fc52c; sha256 83e565f33aa8b3e06c2b4cfc777321dd5f791b347b6611ab86372727e3db9da2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02334. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We compare how often two released language models give outdated numeric values from Norwegian statutes and regulations: NbAiLab/borealis2-26b-a4b-preview (Borealis 2), which the National Library of Norway further trained from google/gemma-4-26B-A4B-it (Gemma 4), and Gemma 4 itself. The study was preregistered (doi: https://doi.org/10.5281/zenodo.23858965 ) and uses the closed norregel benchmark, scored deterministically against a value history. On the 411 of 503 primary items where both models gave an interpretable answer, Borealis 2 alone was outdated on 10 items and Gemma 4 alone on 11 (d = -0.2 percentage points, 95% cluster bootstrap interval [-2.7, +2.5], exact one-sided McNemar p = 0.668). By the preregistered rule this is no effect of interest: the whole interval lies within the smallest effect of interest of +/-10 percentage points. Among the secondary measures, only the share of answers giving a value that took effect after the question date differs after Holm correction; Gemma 4 gave such answers more often. An exploratory comparison of two smaller Borealis models had no discordant pairs and is not informative. The design compares released models and does not isolate the effect of any part of their training. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.02334 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:59:52.334Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).