Metraipolis › archive › dc_30e2ceea70ed

Pixel-space video generation with U-JiT — arXiv:2611.04764

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.04764
License (as archived): CC0-1.0
Archived: 2026-11-04T12:56:37.316Z
SHA-256 of the archived text: 018014523616125fb62afb19313dd94d978492e45bf4cd1922e9f3629223a005
Stable URL of this page: https://metraipolis.com/documento/dc_30e2ceea70ed

How to cite

Pixel-space video generation with U-JiT — arXiv:2611.04764 — arXiv (en), https://arxiv.org/abs/2611.04764 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_30e2ceea70ed (Metraipolis archive dc_30e2ceea70ed, sha256 018014523616125fb62afb19313dd94d978492e45bf4cd1922e9f3629223a005)

BibTeX

@misc{dc30e2ceea70ed,
  title = {Pixel-space video generation with U-JiT — arXiv:2611.04764},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.04764},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_30e2ceea70ed; sha256 018014523616125fb62afb19313dd94d978492e45bf4cd1922e9f3629223a005},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04764. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Latent diffusion and flow matching models are ubiquitous in state-of-the-art generative video models. While highly effective, relying on pretrained autoencoders to compress the data into a lower-dimensional latent space also comes with drawbacks. First, training proceeds in two stages: autoencoder pretraining, with compression and reconstruction objectives, followed by latent generative model training, precluding fully end-to-end training. Second, autoencoders also lead to an inherent loss of signal, and the trade-off between reconstruction quality and generative performance in the latent space is still an active area of research. In this paper we explore the viability of generative video models directly in pixel space. We analyze recent approaches in pixel-space generative modeling of images that rely on basic transformer designs, and identify several shortcomings that limit their effectiveness for the high-dimensional pixel-space video generation. To overcome these limitations, we propose a UNet-like transformer architecture along with improvements to the training recipe and sampling strategy. We conduct controlled experiments, training latent and pixel-space models from scratch on Kinetics-600. For the standard frames-to-video task at 128 resolution, our U-JiT model improves generative FVD by 27% over prior state-of-the-art results, and by 14% against the JiT pixel-space baseline. Scaling to higher 256 resolution, we find the improvement over pixel-space and latent-based models grows as we increase the spatiotemporal compression rate up to $64 \times 64 \times 8$. Finally, we find that U-JiT improves per-frame FID by 27% over latent models and 61% over pixel-space models on class-to-video generation, with respectively 9% and 56% reduction in FVD.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.04764 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T12:33:47.184Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).