Metraipolis › archive › dc_a69448dd1eb3
Transcribe, Translate, and Optimize: Joint Reward Learning for Speech Translation — arXiv:2609.26536
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.26536
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T14:36:49.241Z
- SHA-256 of the archived text
0f615ba678064084c89befc917615c09372c89ba987b5ac5e4a8d4798743d994- Stable URL of this page
https://metraipolis.com/documento/dc_a69448dd1eb3
How to cite
Transcribe, Translate, and Optimize: Joint Reward Learning for Speech Translation — arXiv:2609.26536 — arXiv (en), https://arxiv.org/abs/2609.26536 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a69448dd1eb3 (Metraipolis archive dc_a69448dd1eb3, sha256 0f615ba678064084c89befc917615c09372c89ba987b5ac5e4a8d4798743d994)
BibTeX
@misc{dca69448dd1eb3,
title = {Transcribe, Translate, and Optimize: Joint Reward Learning for Speech Translation — arXiv:2609.26536},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.26536},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a69448dd1eb3; sha256 0f615ba678064084c89befc917615c09372c89ba987b5ac5e4a8d4798743d994},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26536. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
In LLM-based speech translation, transcription-based chain-of-thought (CoT) suffers from a mismatch between reference transcripts used in supervised fine-tuning (SFT) and model-generated transcripts at inference. To address this, we propose joint recognition and translation fine-tuning via group relative policy optimization (GRPO). We score both transcripts and translations, with translation conditioned on model-generated transcripts, and compare three token advantage strategies. Using Qwen2.5-Omni-3B across four languages, we evaluate CoT against direct speech translation (Direct ST) under SFT and GRPO, training on CoVoST 2 and testing on CoVoST 2 and FLEURS. CoT GRPO outperforms Direct ST GRPO by 1.77 and 0.83 average BLEU points on CoVoST 2 and FLEURS. Compared to CoT SFT, GRPO boosts BLEU by 0.82 and 0.67 points and reduces word error rate (WER) by 8.8% and 7.2% relatively. These results highlight reinforcement fine-tuning as an effective method to mitigate the training-inference mismatch, jointly improving recognition and translation. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Audio and Speech Processing (eess.AS). Pagina del lavoro: https://arxiv.org/abs/2609.26536 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T13:46:15.759Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).