Metraipolis › archive › dc_3e59cdf53bdf
Not Quite My Tempo: Voice Activity-aware Speech Synthesis for Lip-Synchronous Dubbing — arXiv:2609.26486
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.26486
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T13:47:16.088Z
- SHA-256 of the archived text
279d26906c4227f8750aed584f0d992b68a933e5dfab71197c1c7dbc8427521c- Stable URL of this page
https://metraipolis.com/documento/dc_3e59cdf53bdf
How to cite
Not Quite My Tempo: Voice Activity-aware Speech Synthesis for Lip-Synchronous Dubbing — arXiv:2609.26486 — arXiv (en), https://arxiv.org/abs/2609.26486 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3e59cdf53bdf (Metraipolis archive dc_3e59cdf53bdf, sha256 279d26906c4227f8750aed584f0d992b68a933e5dfab71197c1c7dbc8427521c)
BibTeX
@misc{dc3e59cdf53bdf,
title = {Not Quite My Tempo: Voice Activity-aware Speech Synthesis for Lip-Synchronous Dubbing — arXiv:2609.26486},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.26486},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3e59cdf53bdf; sha256 279d26906c4227f8750aed584f0d992b68a933e5dfab71197c1c7dbc8427521c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26486. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Automatic lip-synchronous dubbing requires a speech synthesis model to generate alternating voice and silence patterns in the target language that match the timing of the source clip precisely to ensure an optimal viewing experience. Prior works address this problem by conditioning the speech synthesis process on lip movements extracted from the video signal. In this work, we condition the speech generation on a binary voice-activity signal, which has a lightweight representation and can be produced in multiple ways. We show that the model follows the voice-activity signal with high accuracy while maintaining natural prosody and semantically appropriate pause placement within sentences, as demonstrated through extensive objective and subjective evaluations. By randomly masking this condition during training, we make the feature entirely optional during inference, allowing editors to enforce or relax lip-sync constraints when desired. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.AI). Categorie arXiv: Audio and Speech Processing (eess.AS) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.26486 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T13:18:38.320Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).