Metraipolis › archive › dc_05e16c6defea

Emo-LiveSync: Real-Time Universal Lip Synchronization with Explicit Emotion Control — arXiv:2611.09074

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.09074
License (as archived)
CC0-1.0
Archived
2026-11-09T22:45:14.055Z
SHA-256 of the archived text
c95a5c4170a28cbd50797cbb02104a369e862060d445cab00c5ac8ccfdb33a85
Stable URL of this page
https://metraipolis.com/documento/dc_05e16c6defea

How to cite

Emo-LiveSync: Real-Time Universal Lip Synchronization with Explicit Emotion Control — arXiv:2611.09074 — arXiv (en), https://arxiv.org/abs/2611.09074 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_05e16c6defea (Metraipolis archive dc_05e16c6defea, sha256 c95a5c4170a28cbd50797cbb02104a369e862060d445cab00c5ac8ccfdb33a85)
BibTeX
@misc{dc05e16c6defea,
  title = {Emo-LiveSync: Real-Time Universal Lip Synchronization with Explicit Emotion Control — arXiv:2611.09074},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.09074},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_05e16c6defea; sha256 c95a5c4170a28cbd50797cbb02104a369e862060d445cab00c5ac8ccfdb33a85},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09074. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Lip synchronization, the task of generating realistic lip movements synchronized with audio signals, is fundamental to applications such as film dubbing, virtual avatars, and digital content creation. Recent GAN-based methods offer computational efficiency but often lack visual fidelity, while diffusion-based approaches achieve high-quality results at the cost of substantial computational overhead. Moreover, most existing methods are limited to photo-realistic human faces, restricting their applicability to stylized characters and non-human entities. In this paper, we propose an efficient universal lip-sync video generation framework that achieves high-quality results without compromising computational efficiency. We build on LivePortrait, a warp-based portrait animation model, and introduce a training strategy that aligns audio signals to an implicit motion space by distilling knowledge from a pre-trained video-to-motion model. This enables real-time inference while maintaining high visual fidelity, and naturally generalizes to diverse stylistic characters beyond photo-realistic humans. Furthermore, we introduce an expressive control gate that enables explicit manipulation of emotional expressions for content enrichment. Extensive experiments demonstrate that our method achieves state-of-the-art performance in both visual quality and lip-sync accuracy while ensuring real-time inference.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.09074 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-09T22:37:47.219Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).