Metraipolis › archive › dc_80285f59ff90

Factorized Delayed Streams Modeling for LLM-based Streaming ASR — arXiv:2610.04333

Source
arXiv
Original
https://arxiv.org/abs/2610.04333
License (as archived)
CC0-1.0
Archived
2026-10-06T05:28:44.385Z
SHA-256 of the archived text
c17899366c260032547aff6afc09ba712a518732ee59d91f207423ddaef11bd1
Stable URL of this page
https://metraipolis.com/documento/dc_80285f59ff90

How to cite

Factorized Delayed Streams Modeling for LLM-based Streaming ASR — arXiv:2610.04333 — arXiv, https://arxiv.org/abs/2610.04333 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_80285f59ff90 (Metraipolis archive dc_80285f59ff90, sha256 c17899366c260032547aff6afc09ba712a518732ee59d91f207423ddaef11bd1)
BibTeX
@misc{dc80285f59ff90,
  title = {Factorized Delayed Streams Modeling for LLM-based Streaming ASR — arXiv:2610.04333},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.04333},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_80285f59ff90; sha256 c17899366c260032547aff6afc09ba712a518732ee59d91f207423ddaef11bd1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04333. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Delayed Streams Modeling (DSM) enables LLM-based streaming automatic speech recognition (ASR) by aligning acoustic and text streams on a common timeline. DSM adds the padding token <p> and the word-start token <w> to the LLM vocabulary and predicts them together with normal text tokens using the same softmax. We first show that <w> can be removed while maintaining competitive recognition performance. Based on this result, we propose Factorized DSM (F-DSM), which separates the waiting probability for <p> from the distribution over the original LLM vocabulary. This factorization removes ASR-specific tokens from the text prediction space and allows the large-vocabulary softmax to be skipped on waiting steps. Experiments on the Corpus of Spontaneous Japanese and LibriSpeech show that F-DSM achieves better recognition performance than DSM. It also greatly reduces GPU memory use while maintaining similar training throughput, provides a small inference speed improvement through softmax skipping, and reduces the degradation in text-only perplexity observed with DSM.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Audio and Speech Processing (eess.AS) ; Computation and Language (cs.CL); Sound (cs.SD).
Pagina del lavoro: https://arxiv.org/abs/2610.04333 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:20.411Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).