Metraipolis › archive › dc_5ff6dce667d7
Unadapted Multilingual ASR on a Garrusi Kurdish Evaluation Set: A Common-Reference Staged Normalization Analysis — arXiv:2608.16379
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.16379
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T18:59:07.559Z
- SHA-256 of the archived text
2720219d2530ae433af81fcac9afa0cce1c8810cbf99b5ce15c790cd9ac69a05- Stable URL of this page
https://metraipolis.com/documento/dc_5ff6dce667d7
How to cite
Unadapted Multilingual ASR on a Garrusi Kurdish Evaluation Set: A Common-Reference Staged Normalization Analysis — arXiv:2608.16379 — arXiv (en), https://arxiv.org/abs/2608.16379 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5ff6dce667d7 (Metraipolis archive dc_5ff6dce667d7, sha256 2720219d2530ae433af81fcac9afa0cce1c8810cbf99b5ce15c790cd9ac69a05)
BibTeX
@misc{dc5ff6dce667d7,
title = {Unadapted Multilingual ASR on a Garrusi Kurdish Evaluation Set: A Common-Reference Staged Normalization Analysis — arXiv:2608.16379},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.16379},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5ff6dce667d7; sha256 2720219d2530ae433af81fcac9afa0cce1c8810cbf99b5ce15c790cd9ac69a05},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.16379. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Evaluating speech recognition for a Kurdish variety written in a Latin field orthography, using a model that outputs Arabic script, creates a measurement problem before a modelling one: direct scoring treats writing-system differences as recognition errors. Jointly normalizing reference and hypothesis avoids this, but also changes reference tokenization, mixing agreement gains with a change in the scoring denominator. I evaluate MMS-1B-all with the Central Kurdish (ckb) adapter, used as released without adaptation, on 1,722 Garrusi questionnaire segments from five speakers (9,763 reference word tokens; 117.9 minutes). I use a common-reference design: the reference is folded once and fixed at 9,763 tokens, while only the hypothesis representation varies. The raw Arabic-script hypothesis scores 111.70% WER and 100.92% CER, with zero exact word matches. Latin transliteration gives 102.36% WER and 57.89% CER; folding it into the reference's reduced orthography gives 97.85% and 51.20%. Thus RAW-to-FOLDED reduces measured WER by 13.85 points and CER by 49.72 points; folding alone accounts for 4.51 and 6.69 points. Substantial error remains: 14.53% of reference tokens are exact matches, edits are substitution-dominated, and per-segment WER is higher for shorter segments. A Southern Kurdish fine-tuned system (aranemini/southern-kurdish-asr), scored under the same design, performs worse on every speaker (1,703 segments), with 109.56% WER and 55.85% CER. However, 12,330 output characters fall outside the folding table, so these rates must be recomputed against the corrected fixed reference. The MMS output also contains 613 unconverted or unmapped characters, showing that part of the residual error reflects scoring-pipeline limits rather than recognition alone. I will release the fixed reference and segment-level results, subject to source-corpus sharing terms, to support independent checking. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Sound (cs.SD). Pagina del lavoro: https://arxiv.org/abs/2608.16379 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T18:06:57.893Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).