Metraipolis › archive › dc_3f7a4da4e6ee
Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation — arXiv:2609.15313
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.15313
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T23:17:13.786Z
- SHA-256 of the archived text
9e97a173ec1d105b531a948a044df6a3390aad57afcdd17d4d613da99e6b6426- Stable URL of this page
https://metraipolis.com/documento/dc_3f7a4da4e6ee
How to cite
Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation — arXiv:2609.15313 — arXiv (en), https://arxiv.org/abs/2609.15313 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3f7a4da4e6ee (Metraipolis archive dc_3f7a4da4e6ee, sha256 9e97a173ec1d105b531a948a044df6a3390aad57afcdd17d4d613da99e6b6426)
BibTeX
@misc{dc3f7a4da4e6ee,
title = {Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation — arXiv:2609.15313},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.15313},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3f7a4da4e6ee; sha256 9e97a173ec1d105b531a948a044df6a3390aad57afcdd17d4d613da99e6b6426},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.15313. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Autoregressive generation of interleaved text and acoustic tokens is a common approach to spoken-response generation in speech large language models. Although this design enables streaming generation with explicit textual guidance, generated acoustic tokens become part of the context for subsequent text predictions. Given identical speech inputs, we observe markedly lower answer accuracy for the internal text generated in speech-to-text-and-speech (S2TS) mode than for speech-to-text (S2T) responses. We term this discrepancy the \emph{output-mode gap} (OMG). To reduce OMG, we propose \emph{Joint-Output On-Policy Distillation} (JO-OPD), which distills the model's stronger S2T policy into joint generation using student-generated S2TS trajectories. At each text position, the S2T teacher provides soft targets from a text-only projection of the student's preceding outputs, while the student predicts from the corresponding full interleaved history. A preservation objective further regularizes native non-text predictions. Experiments on Step-Audio-2-mini and Baichuan-Audio-Instruct reveal OMG across two interleaved generation architectures. On Step-Audio-2-mini, JO-OPD reduces OMG from 42.87 to 16.26 percentage points on Spoken-MQA and from 29.72 to 13.04 points on speech-rendered GSM8K, with little change in S2T accuracy and substantially larger reductions than matched SFT baselines. ASR-based evaluation further shows a 7.49-point improvement in spoken-answer accuracy on Spoken-MQA.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.CL).
Categorie arXiv: Audio and Speech Processing (eess.AS) ; Computation and Language (cs.CL); Sound (cs.SD).
Pagina del lavoro: https://arxiv.org/abs/2609.15313 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T22:51:10.034Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).