Metraipolis › archive › dc_dbd46ff33e67

Runway Characters: Serving a 57B Video Diffusion Transformer for Real-Time Conversation — arXiv:2611.05642

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.05642
License (as archived): CC0-1.0
Archived: 2026-11-05T18:19:18.777Z
SHA-256 of the archived text: d7244e9fe3d5992f58e9fced7b08ca58b63c8184f1f8db388b427142b97d7361
Stable URL of this page: https://metraipolis.com/documento/dc_dbd46ff33e67

How to cite

Runway Characters: Serving a 57B Video Diffusion Transformer for Real-Time Conversation — arXiv:2611.05642 — arXiv (en), https://arxiv.org/abs/2611.05642 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dbd46ff33e67 (Metraipolis archive dc_dbd46ff33e67, sha256 d7244e9fe3d5992f58e9fced7b08ca58b63c8184f1f8db388b427142b97d7361)

BibTeX

@misc{dcdbd46ff33e67,
  title = {Runway Characters: Serving a 57B Video Diffusion Transformer for Real-Time Conversation — arXiv:2611.05642},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05642},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dbd46ff33e67; sha256 d7244e9fe3d5992f58e9fced7b08ca58b63c8184f1f8db388b427142b97d7361},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05642. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Conversational video characters must generate video as fast as it plays, for one user at a time: at 24 fps, a sustained 166.7 ms per four-frame chunk at batch size one, with only a few chunks of buffering. We describe the system behind Runway Characters, which serves a $\approx$57B-parameter autoregressive video diffusion transformer (four denoising passes per chunk) at 704$\times$1088 and 24 fps on four H100 GPUs with a voice agent in the loop. Each chunk takes 240 attention calls, each reading a 5,788-slot KV cache in which only 757 tokens are new, and 29% of the original step was whole-cache work repeated in every call. We move that work to write time (a ring cache, RoPE and FP8 quantization applied once per write, and a delta rotation when positions reset), fuse the tensor-parallel all-reduce into the FP8 GEMM with in-switch reduction, and give the decoder that turns latents into video frames its own GPU. Without the fused kernel, our optimizations cut the diffusion step from 240.9 to 159.3 ms (1.51$\times$, 24.5 fps sustained) with no detected degradation in lip-sync scores; the fused kernel, rolled back once in production and since re-enabled, reaches 147.4 ms (1.63$\times$, 43% of FP8 peak). In seven weeks of production the worker averaged above 23.5 fps in 99.5% of sessions that streamed for at least a second, and our pipeline takes 0.49 s of a 2.1 s median conversational turn. On the same GPUs, the two open-source real-time video systems we benchmarked, at four denoising passes, miss real time or add over a second of latency (one by estimate), at a third to half of our resolution.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2611.05642 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-05T18:13:18.985Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).