Metraipolis › archive › dc_73b2ee4fc0dd

Poly-InstructTTS: Learning In-the-Wild Expressive Speech Synthesis from Open-Ended Instructions — arXiv:2608.20387

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.20387
License (as archived): CC0-1.0
Archived: 2026-10-18T10:18:16.411Z
SHA-256 of the archived text: 09c590cab7edb4d00c354a145566b0de6ce3c85b93d3b289fa96df3ebf218545
Stable URL of this page: https://metraipolis.com/documento/dc_73b2ee4fc0dd

How to cite

Poly-InstructTTS: Learning In-the-Wild Expressive Speech Synthesis from Open-Ended Instructions — arXiv:2608.20387 — arXiv (en), https://arxiv.org/abs/2608.20387 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_73b2ee4fc0dd (Metraipolis archive dc_73b2ee4fc0dd, sha256 09c590cab7edb4d00c354a145566b0de6ce3c85b93d3b289fa96df3ebf218545)

BibTeX

@misc{dc73b2ee4fc0dd,
  title = {Poly-InstructTTS: Learning In-the-Wild Expressive Speech Synthesis from Open-Ended Instructions — arXiv:2608.20387},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.20387},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_73b2ee4fc0dd; sha256 09c590cab7edb4d00c354a145566b0de6ce3c85b93d3b289fa96df3ebf218545},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.20387. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

While recent text-to-speech (TTS) models achieve high naturalness, controlling fine-grained expression via natural-language instructions remains challenging. We introduce Poly- InstructTTS, which learns expressive speech from open-ended instructions using in-the-wild audiovisual data. We build a scalable multi-modal pipeline to construct a 1,000-hour instruction-annotated corpus covering 1,000+ fine-grained emotions and styles. The framework uses a prompt-free GPT with attribute-based thinking tokens, followed by a flow-matching module that injects timbre from a reference audio. We also present a speaker fine-tuning procedure to transfer instruction control to specific speakers while preserving persona. We further extend InstructTTSEval with broader tasks. Experiments show that Poly-InstructTTS delivers strong performance in instruction adherence and expressiveness. Audio demos and the expanded testset are available on our project page.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-24 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.20387 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T10:17:37.565Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).