Metraipolis › archive › dc_6fa00cbdb33c

Vorch-Streamer: Extending Human Audio-Visual Generation to Real-Time Long-Form Streaming — arXiv:2608.05663

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.05663
License (as archived): CC0-1.0
Archived: 2026-10-31T10:36:04.115Z
SHA-256 of the archived text: 1f957ee74aa70f375805c2b556e5d8217a015abc056637b192eb8ad39dee685d
Stable URL of this page: https://metraipolis.com/documento/dc_6fa00cbdb33c

How to cite

Vorch-Streamer: Extending Human Audio-Visual Generation to Real-Time Long-Form Streaming — arXiv:2608.05663 — arXiv (en), https://arxiv.org/abs/2608.05663 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6fa00cbdb33c (Metraipolis archive dc_6fa00cbdb33c, sha256 1f957ee74aa70f375805c2b556e5d8217a015abc056637b192eb8ad39dee685d)

BibTeX

@misc{dc6fa00cbdb33c,
  title = {Vorch-Streamer: Extending Human Audio-Visual Generation to Real-Time Long-Form Streaming — arXiv:2608.05663},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.05663},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6fa00cbdb33c; sha256 1f957ee74aa70f375805c2b556e5d8217a015abc056637b192eb8ad39dee685d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.05663. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Real-time long-form avatar audio-video generation requires causal, continuous synthesis while maintaining audiovisual synchronization and visual consistency. Adapting a pretrained bidirectional model to this setting presents two key dilemmas. First, autoregressively reusing generated blocks as context creates exposure bias, causing errors and visual drift to accumulate over long rollouts. Second, a global speech utterance does not indicates a causal generator which portion should be spoken next when only limited local audio-video context is available. We present Vorch-Streamer, a post-training framework that addresses these challenges and enables real-time long-form Text-to-Audio-Video (T2AV) streaming. We construct a synthetic corpus of 80K avatar clips spanning 12-21 seconds and first train a causal generator with mixed Teacher Forcing and Diffusion Forcing. We then apply long-horizon Self Forcing with DMD distillation, exposing the model to its own rollout distribution while preserving the quality of the pretrained bidirectional teacher. To explicitly control speech progression, an external language model predicts discrete 25-Hz speech-planning tokens, whose continuous features condition the audio diffusion branch and align each causal block with the content it should speak. With bounded causal context and four-step denoising, Vorch-Streamer jointly generates audio and video from text at 27.12 FPS, exceeding the 24-FPS real-time playback rate while maintaining competitive audio-lip synchronization and strong identity preservation over long-form generation.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-07 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Sound (cs.SD).
Pagina del lavoro: https://arxiv.org/abs/2608.05663 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T10:23:30.992Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).