Metraipolis › archive › dc_f1bd88c49ebe

Luna-TTS Family Technical Report — arXiv:2608.11593

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.11593
License (as archived)
CC0-1.0
Archived
2026-10-19T06:07:13.202Z
SHA-256 of the archived text
2337b471d2a64130d2fda049f6e6f8a2fb61a84bf7e24f237b51be6ba6bfd7b5
Stable URL of this page
https://metraipolis.com/documento/dc_f1bd88c49ebe

How to cite

Luna-TTS Family Technical Report — arXiv:2608.11593 — arXiv (en), https://arxiv.org/abs/2608.11593 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f1bd88c49ebe (Metraipolis archive dc_f1bd88c49ebe, sha256 2337b471d2a64130d2fda049f6e6f8a2fb61a84bf7e24f237b51be6ba6bfd7b5)
BibTeX
@misc{dcf1bd88c49ebe,
  title = {Luna-TTS Family Technical Report — arXiv:2608.11593},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11593},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f1bd88c49ebe; sha256 2337b471d2a64130d2fda049f6e6f8a2fb61a84bf7e24f237b51be6ba6bfd7b5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11593. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Modern text-to-speech (TTS) is dominated by autoregressive (AR) codec language models, whose left-to-right decoding brings latency that grows with utterance length, error accumulation along the committed prefix, and an artificial generation order imposed on the Residual Vector Quantization (RVQ) token grid. We propose Luna-TTS Family, diffusion-language-model-based TTS systems pretrained on 1 million hours of speech across Chinese, English, Japanese, and Korean. The family is built by progressive adaptation of a pretrained AR text LLM, from causal to bidirectional and finally to block-causal attention, and comprises two variants sharing a single tokenizer, data pipeline, and 0.6B backbone lineage. Luna-TTS is fully non-autoregressive: it generates the entire RVQ token grid in a fixed number of parallel refinement steps, with zero-shot voice cloning and speech editing arising natively as infilling. Luna-TTS Realtime, derived by continual training, is autoregressive over blocks of 32 codec frames (1.28s) while denoising each block in parallel; it supports KV-cached blockwise generation and incremental audio delivery, achieving an end-to-end RTF of 0.0240 and 41.6 ms local first-block latency under the warmed serving protocol. An annealed fine-tuning stage adds explicit control over emotion and non-verbal vocalizations (NVVs), and a reinforcement-learning stage applies GRPO with policy ratios computed over the realized denoising trajectory. On Seed-TTS-Eval, Luna-TTS achieves the best results on all four metrics among compared open-source and commercial systems (0.73 CER / 79.7 SIM on test-zh, 1.49 WER / 76.8 SIM on test-en); on the harder in-the-wild CV3-Eval, it posts the lowest Mandarin and English error rates in our comparison. Against leading commercial systems, it achieves the best results on most objective, model-based, and human-rated metrics for NVV and emotion control.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in eess.AS).
Categorie arXiv: Sound (cs.SD) ; Audio and Speech Processing (eess.AS).
Pagina del lavoro: https://arxiv.org/abs/2608.11593 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:56:37.848Z dalla pagina https://arxiv.org/catchup/eess.AS/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).