Metraipolis › archive › dc_f62756adfa32

V-RAE: Rethinking Video Latent Spaces for Generation — arXiv:2608.13556

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.13556
License (as archived)
CC0-1.0
Archived
2026-10-19T04:56:00.243Z
SHA-256 of the archived text
b1460c9921ef8fe65a787c1a7b46ebbd8cdfc0d72a0e841b370f1414c124e911
Stable URL of this page
https://metraipolis.com/documento/dc_f62756adfa32

How to cite

V-RAE: Rethinking Video Latent Spaces for Generation — arXiv:2608.13556 — arXiv (en), https://arxiv.org/abs/2608.13556 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f62756adfa32 (Metraipolis archive dc_f62756adfa32, sha256 b1460c9921ef8fe65a787c1a7b46ebbd8cdfc0d72a0e841b370f1414c124e911)
BibTeX
@misc{dcf62756adfa32,
  title = {V-RAE: Rethinking Video Latent Spaces for Generation — arXiv:2608.13556},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.13556},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f62756adfa32; sha256 b1460c9921ef8fe65a787c1a7b46ebbd8cdfc0d72a0e841b370f1414c124e911},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.13556. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Latent video generation relies on autoencoders to define a compact space in which generative models operate. Although video autoencoder architectures have evolved substantially, their latent spaces are still optimized primarily for pixel-level reconstruction and provide limited high-level semantic organization. A reconstruction-optimal latent space, however, need not be well suited to generative modeling. We propose V-RAE, a video representation autoencoder that builds compact generative latents on top of frozen vision foundation model representations. A lightweight temporal pooling module removes temporal redundancy while preserving semantic structure, and a video decoder reconstructs continuous motion from the compressed features. We evaluate V-RAE with four representative frozen encoders on video reconstruction, semantic probing, and class-conditional generation. V-RAE achieves 2.13 rFVD on K600, outperforming all evaluated large-scale pretrained video VAEs. Its latents retain substantially more semantic information than conventional video tokenizer latents. Under matched generation settings, our best variant achieves gFVD scores of 117.86 and 19.16 on UCF101 and K600, respectively, while converging up to 6x faster}. We further show that reconstruction quality alone is insufficient to characterize generative utility and introduce tFVD, a temporal-coherence diagnostic that correlates more reliably with downstream generation quality. Beyond video generation, V-RAE also improves future video prediction on Cityscapes over the Wan 2.2 VAE latent space under matched prediction settings. Taken together, the experiments show that frozen semantic representations can support video reconstruction, generation, and predictive modeling. The project page: this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-14 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.13556 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T04:46:30.256Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).