Metraipolis › archive › dc_802807bf0ee1
On-Policy Self-Distillation for Multi-Dialect ASR: Mastering Dialects, Retaining Mandarin — arXiv:2608.11898
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.11898
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T06:07:11.572Z
- SHA-256 of the archived text
6d9c9885447ce44c3194d83cd12e180bb9a4c91ff172b9767c5f73103b1e7eea- Stable URL of this page
https://metraipolis.com/documento/dc_802807bf0ee1
How to cite
On-Policy Self-Distillation for Multi-Dialect ASR: Mastering Dialects, Retaining Mandarin — arXiv:2608.11898 — arXiv (en), https://arxiv.org/abs/2608.11898 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_802807bf0ee1 (Metraipolis archive dc_802807bf0ee1, sha256 6d9c9885447ce44c3194d83cd12e180bb9a4c91ff172b9767c5f73103b1e7eea)
BibTeX
@misc{dc802807bf0ee1,
title = {On-Policy Self-Distillation for Multi-Dialect ASR: Mastering Dialects, Retaining Mandarin — arXiv:2608.11898},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.11898},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_802807bf0ee1; sha256 6d9c9885447ce44c3194d83cd12e180bb9a4c91ff172b9767c5f73103b1e7eea},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11898. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Recent large-scale ASR models already achieve strong Mandarin recognition accuracy and have some ability to recognize Chinese dialects. However, their dialect recognition accuracy is still limited in real-world speech. Direct dialect adaptation can lower dialect CER, but it may also raise Mandarin CER. We therefore study how to adapt a capable ASR model to improve multi-dialect recognition without degrading Mandarin recognition. We adopt an adaptation pipeline where continual pre-training (CPT) and dialect supervised fine-tuning (SFT) provide a strong foundation, and On-Policy Self-Distillation (OPSD) serves as the final refinement. OPSD addresses the train--test mismatch in autoregressive ASR by training the student model on its own decoded prefixes while a frozen teacher, conditioned on the reference transcript as privileged context, provides soft token-level targets. This replaces hard cross-entropy updates on dialect data with distillation, preserving Mandarin ability while refining dialect recognition. We instantiate the framework with Qwen3-ASR-1.7B and evaluate it on public and internal Mandarin and dialect test sets. Under matched refinement data and schedule, OPSD improves dialect recognition without raising Mandarin CER, whereas continued teacher-forced fine-tuning increases Mandarin CER. We will release the model weights and evaluation scripts. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in eess.AS). Categorie arXiv: Audio and Speech Processing (eess.AS). Pagina del lavoro: https://arxiv.org/abs/2608.11898 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T05:56:37.845Z dalla pagina https://arxiv.org/catchup/eess.AS/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).