Metraipolis › archive › dc_ed3fa51ba2d2
Decoupled Self-Forcing Distillation for Streaming Talking Head Generation — arXiv:2609.10317
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.10317
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T11:00:01.141Z
- SHA-256 of the archived text
eeceddced135cfe498e39c166469469386eea24053b54c8de540f9a50e76bf17- Stable URL of this page
https://metraipolis.com/documento/dc_ed3fa51ba2d2
How to cite
Decoupled Self-Forcing Distillation for Streaming Talking Head Generation — arXiv:2609.10317 — arXiv (en), https://arxiv.org/abs/2609.10317 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ed3fa51ba2d2 (Metraipolis archive dc_ed3fa51ba2d2, sha256 eeceddced135cfe498e39c166469469386eea24053b54c8de540f9a50e76bf17)
BibTeX
@misc{dced3fa51ba2d2,
title = {Decoupled Self-Forcing Distillation for Streaming Talking Head Generation — arXiv:2609.10317},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.10317},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ed3fa51ba2d2; sha256 eeceddced135cfe498e39c166469469386eea24053b54c8de540f9a50e76bf17},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.10317. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Streaming talking-head generation produces each frame as its driving audio arrives, yet fidelity and efficiency have so far pulled in opposite directions: end-to-end methods condition a video diffusion model on audio directly and achieve high quality but only at large scale, while cheaper two-stage methods generate an intermediate motion representation and trail in fidelity. We argue the cost of the former lies in the target of fusion: the video latent is dominated by identity, appearance and background, none of which audio bears on, so coupling audio to every pixel blurs detail and wastes capacity. We instead fuse conditions in a low-dimensional identity-disentangled motion space, routing audio and motion captions by their temporal granularity, and generate motion latents with a small causal autoregressive transformer that a pretrained diffusion renderer turns into video. Conditions thus control video transitively, and high fidelity no longer requires a large backbone. Streaming this decomposition needs both models to be causal, and the exposure-bias problem could be solved by self-forcing given a bidirectional teacher. But there is no such teacher in motion space. Our decoupled self-forcing distillation resolves both models under one frozen teacher: conditioned on motion, it distills the renderer into a block-causal student; unconditionally, it scores rendered rollouts against real videos, supervising motion by the video it produces. This lifts the fidelity ceiling from the motion generator onto the stronger renderer. The two models run as parallel causal streams, reaching 15.4 FPS at 1.3 s latency with no quality degradation. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-10 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.10317 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T10:02:34.373Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).