Metraipolis › archive › dc_dca67cb14c9c
Modeling, Scaling, and Decoding: Optimizing Controllable Speech Generation with Nonverbal Vocalizations — arXiv:2609.14231
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.14231
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T22:50:59.168Z
- SHA-256 of the archived text
945b744a3e610db9393c539db35fe83b8b78cb9bcc9cfb84c5f42adbb76e33e9- Stable URL of this page
https://metraipolis.com/documento/dc_dca67cb14c9c
How to cite
Modeling, Scaling, and Decoding: Optimizing Controllable Speech Generation with Nonverbal Vocalizations — arXiv:2609.14231 — arXiv (en), https://arxiv.org/abs/2609.14231 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dca67cb14c9c (Metraipolis archive dc_dca67cb14c9c, sha256 945b744a3e610db9393c539db35fe83b8b78cb9bcc9cfb84c5f42adbb76e33e9)
BibTeX
@misc{dcdca67cb14c9c,
title = {Modeling, Scaling, and Decoding: Optimizing Controllable Speech Generation with Nonverbal Vocalizations — arXiv:2609.14231},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.14231},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dca67cb14c9c; sha256 945b744a3e610db9393c539db35fe83b8b78cb9bcc9cfb84c5f42adbb76e33e9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.14231. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Controllable synthesis of nonverbal vocalizations (NVVs) is es- sential for natural and expressive speech, but remains challeng- ing due to their acoustic diversity and imbalanced distribution in existing corpora. To address these challenges, we develop an NVV-aware DiTAR system that models continuous speech latents, encodes the 16 target NVV categories as dedicated to- kens, and adapts stop prediction to distinguish mid-utterance vocalizations from utterance boundaries. Training begins with large-scale bilingual pre-training on diverse NVV speech, fol- lowed by continued supervised fine-tuning on a corpus en- hanced through targeted synthetic augmentation and frequency- aware rebalancing. At inference time, we select the acoustic prompt, tune the LM-guidance and noise-injection scales, and apply Best-of-N sampling with multi-metric selection to re- duce generation failures. The final system achieves an official weighted bilingual score of 62.786, ranking first in Mandarin, second in English, and first overall among participating systems in Track 2 of the ISCSLP 2026 NVVSpeech Challenge. Ab- lation studies show that targeted augmentation benefits under- represented NVV categories the most, while robust candidate selection requires balancing NVV correctness, lexical fidelity, and perceptual quality. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.AI). Categorie arXiv: Audio and Speech Processing (eess.AS) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.14231 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:29:03.641Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).