Metraipolis › archive › dc_c93be9c2b7ca
AnyTalk: Speech Animation for Arbitrary Characters Leveraging a Video Generation Model — arXiv:2608.16143
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.16143
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T00:08:33.907Z
- SHA-256 of the archived text
113d26c36702366b3a8aec0709ad4f4642cb2fc551a95a5541970b561c1ea527- Stable URL of this page
https://metraipolis.com/documento/dc_c93be9c2b7ca
How to cite
AnyTalk: Speech Animation for Arbitrary Characters Leveraging a Video Generation Model — arXiv:2608.16143 — arXiv (en), https://arxiv.org/abs/2608.16143 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c93be9c2b7ca (Metraipolis archive dc_c93be9c2b7ca, sha256 113d26c36702366b3a8aec0709ad4f4642cb2fc551a95a5541970b561c1ea527)
BibTeX
@misc{dcc93be9c2b7ca,
title = {AnyTalk: Speech Animation for Arbitrary Characters Leveraging a Video Generation Model — arXiv:2608.16143},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.16143},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c93be9c2b7ca; sha256 113d26c36702366b3a8aec0709ad4f4642cb2fc551a95a5541970b561c1ea527},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.16143. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We present AnyTalk, a novel method for generating 3D speech animations for arbitrary characters without requiring any animation data. While existing audio-driven 3D speech animation methods rely on character-specific training data or laborious rigging/re-meshing, AnyTalk circumvents these limitations by leveraging recent video diffusion models trained on extensive video datasets. We first adapt a pre-trained video diffusion model to a target character through our Character-specific Fine-tuning (\textit{CsF}) technique. By fine-tuning on rendered images of the 3D character paired with zeroed-out audio embeddings (representing "no motion"), we eliminate the need for animation data while preserving the motion prior of large-scale video diffusion model. We then uplift the resulting talking-head video into a 3D speech animation by estimating blendshape parameters through a proposed optimization process. AnyTalk enables lip-synced animations across diverse face meshes and blendshape configurations, significantly reducing manual effort and data requirements. We further enhance usability by distilling AnyTalk into a streamlined network, $\text{AnyTalk}_{RT}$, thereby enabling real-time performance. By leveraging talking-head video generation, our method broadens access to audio-driven speech animation technology for arbitrary characters. The code is publicly available at this https URL .
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.MM).
Categorie arXiv: Graphics (cs.GR) ; Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD).
Pagina del lavoro: https://arxiv.org/abs/2608.16143 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T23:08:37.287Z dalla pagina https://arxiv.org/catchup/cs.MM/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).