Metraipolis › archive › dc_eeaa9041eada

Steering intonational tunes in text-to-speech synthesis — arXiv:2610.27403

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.27403
License (as archived): CC0-1.0
Archived: 2026-10-21T07:09:18.207Z
SHA-256 of the archived text: 183af624809f6663ac271c33e5d418b03b37986d3b2b8687cb5354a4bf93b48c
Stable URL of this page: https://metraipolis.com/documento/dc_eeaa9041eada

How to cite

Steering intonational tunes in text-to-speech synthesis — arXiv:2610.27403 — arXiv (en), https://arxiv.org/abs/2610.27403 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_eeaa9041eada (Metraipolis archive dc_eeaa9041eada, sha256 183af624809f6663ac271c33e5d418b03b37986d3b2b8687cb5354a4bf93b48c)

BibTeX

@misc{dceeaa9041eada,
  title = {Steering intonational tunes in text-to-speech synthesis — arXiv:2610.27403},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.27403},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_eeaa9041eada; sha256 183af624809f6663ac271c33e5d418b03b37986d3b2b8687cb5354a4bf93b48c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.27403. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Text-to-speech systems built on residual vector quantization often generate audio in two stages: a transformer backbone emits the first level of each frame's token stack, and a smaller depth model generates the rest autoregressively. We ask which components of such a model are responsible for prosodic intonation, more precisely for generating the English nuclear tune (for instance, the falling pitch melody that distinguishes an assertion from a question's rising melody), and whether interventions on those components can be used to control intonation. Prompt-based control mostly fails; only a final question mark is a reliable lever. Interchange interventions in CSM and S2 locate the tune: the backbone's hidden state carries it to the waveform through the depth model, and the first level of the stack carries little or none of it. Steering the backbone state with a difference-of-means vector computed on new material generated with a final period or question mark moves the tune, but the push toward an assertion degrades the voice. Steering vectors computed instead from the backbone states associated with human imitations of theoretically-specified tunes, whose text carries no punctuation, and applied inside the generation loop, move most recordings to the other condition in both systems (even when fitted without the speaker being steered), at a small cost in predicted naturalness. These imitation-derived vectors are aligned with the one obtained from the two punctuated conditions far beyond a shuffled-label null, so the backbone state of a punctuated prompt already represents intonation. The steered recordings broadly become falling or rising, according to the intent, but the edge tone moves more coherently than the pitch accent, and the target tune cannot be selected precisely beyond this broad tendency.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Sound (cs.SD) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.27403 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:58.169Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).