Metraipolis › archive › dc_4f890e855504

Watch Your Speech: Text-aware Video-to-Speech Synthesis with Textual Conditioning — arXiv:2610.01012

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.01012
License (as archived)
CC0-1.0
Archived
2026-10-06T19:12:06.434Z
SHA-256 of the archived text
1ee647b49dce6d3d4a36144c445ea801c725a16d570b7e5bb81850ebf628dd58
Stable URL of this page
https://metraipolis.com/documento/dc_4f890e855504

How to cite

Watch Your Speech: Text-aware Video-to-Speech Synthesis with Textual Conditioning — arXiv:2610.01012 — arXiv (en), https://arxiv.org/abs/2610.01012 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4f890e855504 (Metraipolis archive dc_4f890e855504, sha256 1ee647b49dce6d3d4a36144c445ea801c725a16d570b7e5bb81850ebf628dd58)
BibTeX
@misc{dc4f890e855504,
  title = {Watch Your Speech: Text-aware Video-to-Speech Synthesis with Textual Conditioning — arXiv:2610.01012},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.01012},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4f890e855504; sha256 1ee647b49dce6d3d4a36144c445ea801c725a16d570b7e5bb81850ebf628dd58},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01012. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Video-to-speech synthesis aims to generate natural-sounding speech from silent talking-face videos while ensuring phonetic accuracy. A fundamental challenge in this task is the inherent one-to-many mapping problem, where visual dynamics often lack sufficient information to uniquely determine the corresponding utterance. To address this, we propose Watch Your Speech (WYS), a video-to-speech synthesis framework that incorporates textual conditioning as an explicit linguistic cue to mitigate visual ambiguity. Our framework features an attention-based embedding fusion module that synergistically integrates textual context with video sequences, coupled with a conditional flow matching objective for high-fidelity speech generation. Extensive experiments on the LRS2 and LRS3 datasets demonstrate that WYS achieves superior performance, establishing new state-of-the-art results in audio-visual synchronization (LSE-C/D) while maintaining highly competitive textual accuracy (WER). Subjective evaluations further confirm that our model generates speech with near-human naturalness, validating the effectiveness of textual conditioning in content-controlled video-to-speech synthesis. Project page: this https URL

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-10-02 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Sound (cs.SD); Audio and Speech Processing (eess.AS).
Pagina del lavoro: https://arxiv.org/abs/2610.01012 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T17:37:36.720Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).