Metraipolis › archive › dc_978d4274e072
Rethinking Speech Codecs: From Compression to Autoregressive Generative Modeling — arXiv:2609.04237
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.04237
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T23:52:35.593Z
- SHA-256 of the archived text
619d647d69940766d73e9e9ff21fcdf0f5b45ff30fc7de1125f512bdf31ac696- Stable URL of this page
https://metraipolis.com/documento/dc_978d4274e072
How to cite
Rethinking Speech Codecs: From Compression to Autoregressive Generative Modeling — arXiv:2609.04237 — arXiv (en), https://arxiv.org/abs/2609.04237 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_978d4274e072 (Metraipolis archive dc_978d4274e072, sha256 619d647d69940766d73e9e9ff21fcdf0f5b45ff30fc7de1125f512bdf31ac696)
BibTeX
@misc{dc978d4274e072,
title = {Rethinking Speech Codecs: From Compression to Autoregressive Generative Modeling — arXiv:2609.04237},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.04237},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_978d4274e072; sha256 619d647d69940766d73e9e9ff21fcdf0f5b45ff30fc7de1125f512bdf31ac696},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04237. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Recent advances in speech language models leverage discrete speech representations from pretrained codecs to enable scalable training and generation. However, existing codecs are primarily optimized for compression without accounting for the autoregressive nature of language model training, resulting in suboptimal performance when modeling compressed speech tokens. In this work, we revisit speech discretization from a generative modeling perspective and propose a novel framework that explicitly aligns speech tokenization with autoregressive training. Our approach introduces autoregressive-compatible constraints during codec training, encouraging token sequences that exhibit temporal consistency and predictability. In addition, we propose a heterogeneous downsampling strategy for different layers of speech tokens, distinguishing semantic from acoustic layers, to improve the alignment between semantic tokens and corresponding textual content. Extensive experiments across multiple benchmarks demonstrate that our method bridges the gap between speech compression and generative modeling, enabling more effective continued pretraining of existing language models on speech data. The approach consistently improves performance across multiple codecs, validating its generality and applicability to diverse speech modeling scenarios. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-07 in cs.MM). Categorie arXiv: Audio and Speech Processing (eess.AS) ; Multimedia (cs.MM); Sound (cs.SD). Pagina del lavoro: https://arxiv.org/abs/2609.04237 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T23:22:22.347Z dalla pagina https://arxiv.org/catchup/cs.MM/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).