Metraipolis › archive › dc_dc5e4192ec70
Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM — arXiv:2609.17435
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.17435
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T19:19:29.927Z
- SHA-256 of the archived text
150bec0ed69954d8804e95d55eeb7ce98557f69ac22e433c8f569161e868089a- Stable URL of this page
https://metraipolis.com/documento/dc_dc5e4192ec70
How to cite
Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM — arXiv:2609.17435 — arXiv (en), https://arxiv.org/abs/2609.17435 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dc5e4192ec70 (Metraipolis archive dc_dc5e4192ec70, sha256 150bec0ed69954d8804e95d55eeb7ce98557f69ac22e433c8f569161e868089a)
BibTeX
@misc{dcdc5e4192ec70,
title = {Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM — arXiv:2609.17435},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.17435},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dc5e4192ec70; sha256 150bec0ed69954d8804e95d55eeb7ce98557f69ac22e433c8f569161e868089a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17435. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We submit MéTRON-FR, a 125M GPT-2 pretrained on 92.47M words of French, to the BabyLM 2026 Strict track. It scores 85.97 +/- 0.17% on QFrBLiMP (a native Quebec-French benchmark of grammatical minimal pairs) and 62.80% on the BabyLM-weighted leaderboard. A cross-lingual GLUE (General Language Understanding Evaluation) protocol that combines French task-data translation with rank-16 LoRA (Low-Rank Adaptation) produces a sharp task-type gradient: relational tasks gain measurably, while world-knowledge tasks regress. Bilingual Lexicon Induction aligns the French embeddings to GPT-2 at p@1 = 68.84 +/- 8.61%, 18X above chance, suggesting cross-lingual alignment tracks acquired grammatical competence rather than training duration. An ablation study shows that single-token zero-shot scoring is dominated by tokenizer and template artifacts at the child scale, motivating tokenizer-swap sensitivity, placebo-controlled prompting, and native-language minimal-pair benchmarks as standard diagnostics. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.17435 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T18:32:32.875Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).