Metraipolis › archive › dc_eafd4f08a323
TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context — arXiv:2609.08703
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.08703
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T12:58:17.025Z
- SHA-256 of the archived text
cc44b0a4f3af2793418104dd633fbb5ad54008c2211fb423d530dd31dda11e03- Stable URL of this page
https://metraipolis.com/documento/dc_eafd4f08a323
How to cite
TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context — arXiv:2609.08703 — arXiv (en), https://arxiv.org/abs/2609.08703 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_eafd4f08a323 (Metraipolis archive dc_eafd4f08a323, sha256 cc44b0a4f3af2793418104dd633fbb5ad54008c2211fb423d530dd31dda11e03)
BibTeX
@misc{dceafd4f08a323,
title = {TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context — arXiv:2609.08703},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.08703},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_eafd4f08a323; sha256 cc44b0a4f3af2793418104dd633fbb5ad54008c2211fb423d530dd31dda11e03},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.08703. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Text-to-speech systems often face a trade-off between natural prosody and efficient inference: higher perceptual quality usually comes at increased computational cost and latency. We present TontaubeV1, a model that preserves natural prosody while enabling streaming from a single consumer GPU. Speech is encoded by the hierarchical DualCodec representation at 12.5 Hz, which separates a semantic stream from successive acoustic refinements. Our design assumes that prosodic structure is largely established when the semantic stream is generated, and allocates capacity accordingly: a Qwen3-1.7B-derived transformer predicts that stream and thereby the utterance duration, while three progressively smaller Qwen3-0.6B-derived transformers each add one acoustic refinement. Text is tokenized per character rather than by subword. Paired text and audio markers at shared positions support long-form generation with bounded context, and overlapping DualCodec reconstructions are mapped into the VibeVoice acoustic latent space and decoded causally, enabling streaming despite DualCodec's noncausal decoder. The model accepts up to one minute of reference audio for voice conditioning and is designed primarily for English and German, with additional multilingual support. The four predictors total 2.9B parameters; on a single RTX 5090 the streaming path reaches about 200 ms to first audio. In separate non-streaming measurements, the end-to-end real-time factor (RTF) is 0.08 for one input and the aggregate RTF is 0.02 across eight concurrent inputs. In a blinded study, 40 listeners preferred TontaubeV1's prosody over Fish Audio S2 Pro in 70.25% of 400 comparisons (95% CI: 64.2-75.7%). On an LLM-as-a-judge benchmark, it matches ElevenLabs Flash v2.5 and outperforms the April 2026 Gradium API and Cartesia Sonic 3 on prosody. The weights are released on Hugging Face under the Tontaube Community Model License 1.0. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CL). Categorie arXiv: Sound (cs.SD) ; Computation and Language (cs.CL); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.08703 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T12:30:25.362Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).