Metraipolis › archive › dc_d023d424ba57

Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder — arXiv:2608.11650

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.11650
License (as archived)
CC0-1.0
Archived
2026-10-19T05:30:57.315Z
SHA-256 of the archived text
8d1e08d21a5364c508e60ece0fa771bab6bc8b01c51ada48c93953577d080296
Stable URL of this page
https://metraipolis.com/documento/dc_d023d424ba57

How to cite

Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder — arXiv:2608.11650 — arXiv (en), https://arxiv.org/abs/2608.11650 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d023d424ba57 (Metraipolis archive dc_d023d424ba57, sha256 8d1e08d21a5364c508e60ece0fa771bab6bc8b01c51ada48c93953577d080296)
BibTeX
@misc{dcd023d424ba57,
  title = {Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder — arXiv:2608.11650},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11650},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d023d424ba57; sha256 8d1e08d21a5364c508e60ece0fa771bab6bc8b01c51ada48c93953577d080296},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11650. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recent advances in zero-shot text-to-speech (TTS) have substantially improved speech quality and voice cloning fidelity. However, many zero-shot TTS systems still depend on audio prompt transcripts at inference time. This dependency limits cross-lingual voice cloning, since in-the-wild reference audio is often untranscribed. In this technical report, we present Confucius4-TTS, a multilingual zero-shot TTS system that supports 14 languages and performs both intra-lingual and cross-lingual reference cloning without requiring transcripts of audio prompts. Confucius4-TTS follows a two-stage architecture, consisting of text-to-semantic (T2S) and semantic-to-acoustic (S2A) modules. The LLM-based T2S module uses a learnable speaker encoder to extract timbre features from self-supervised speech representations, and the conditional flow-matching S2A module converts the predicted semantic tokens into mel-spectrograms. The same model also supports continuation cloning when a reference transcript is available. Confucius4-TTS is trained on large-scale multilingual speech data. It achieves high intelligibility and speaker similarity on public benchmarks. On the CV3-Eval cross-lingual benchmark, Confucius4-TTS obtains an average WER of 3.73% across six directions. On our internal cross-lingual set, it achieves the best average overall rank in human evaluation among recent open-source and commercial systems. We release code, model checkpoints, and demos at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.CL).
Categorie arXiv: Sound (cs.SD) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.11650 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:15:39.247Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).