Metraipolis › archive › dc_ba5b8a259e1a

StreamAlign: Streaming Text-Aligned Speech Tokenization — arXiv:2609.09719

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.09719
License (as archived)
CC0-1.0
Archived
2026-10-16T07:51:18.216Z
SHA-256 of the archived text
de4e55a4ef3e60c34f120e7476d56ef2bc1ec226a8228a835e52e204321b3822
Stable URL of this page
https://metraipolis.com/documento/dc_ba5b8a259e1a

How to cite

StreamAlign: Streaming Text-Aligned Speech Tokenization — arXiv:2609.09719 — arXiv (en), https://arxiv.org/abs/2609.09719 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ba5b8a259e1a (Metraipolis archive dc_ba5b8a259e1a, sha256 de4e55a4ef3e60c34f120e7476d56ef2bc1ec226a8228a835e52e204321b3822)
BibTeX
@misc{dcba5b8a259e1a,
  title = {StreamAlign: Streaming Text-Aligned Speech Tokenization — arXiv:2609.09719},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.09719},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ba5b8a259e1a; sha256 de4e55a4ef3e60c34f120e7476d56ef2bc1ec226a8228a835e52e204321b3822},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09719. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Text-aligned speech tokenization methods have emerged to better align speech tokens with LLM token spaces, enabling more effective utilization of pretrained LLMs. However, they rely on offline automatic speech recognition (ASR), leading to two key limitations: (i) the need for complete utterances before tokenization, precluding real-time streaming, and (ii) vocabulary mismatch between ASR and LLMs, which reduces acoustic granularity from the subword to the word level. We introduce StreamAlign, a text-aligned speech tokenization framework that enables streaming tokenization for real-time speech-text joint modeling. StreamAlign performs online speech-text alignment by combining character-level RNN-Transducer alignment with word-level ASR guidance, mitigating ASR-LLM vocabulary mismatch while preserving recognition accuracy. A proactive word boundary classifier anticipates word completion at chunk boundaries, reducing tokenization latency from 560 ms to 270 ms. On LibriSpeech, StreamAlign achieves the lowest WER and highest UTMOS among evaluated tokenizers. Furthermore, StreamAlign-SLM, a spoken language model trained on StreamAlign units, outperforms other end-to-end spoken language models in speech continuation while achieving the strongest overall consistency on SALMon and spoken StoryCloze.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-10 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Sound (cs.SD); Audio and Speech Processing (eess.AS).
Pagina del lavoro: https://arxiv.org/abs/2609.09719 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T07:24:00.890Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).