Metraipolis › archive › dc_44c03e951a48
PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation — arXiv:2609.38597
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.38597
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T14:45:24.346Z
- SHA-256 of the archived text
7c0763ead7fff32030097d26d7d4f5832a87103c2118e1c088266e5de13b5957- Stable URL of this page
https://metraipolis.com/documento/dc_44c03e951a48
How to cite
PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation — arXiv:2609.38597 — arXiv (en), https://arxiv.org/abs/2609.38597 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_44c03e951a48 (Metraipolis archive dc_44c03e951a48, sha256 7c0763ead7fff32030097d26d7d4f5832a87103c2118e1c088266e5de13b5957)
BibTeX
@misc{dc44c03e951a48,
title = {PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation — arXiv:2609.38597},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.38597},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_44c03e951a48; sha256 7c0763ead7fff32030097d26d7d4f5832a87103c2118e1c088266e5de13b5957},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38597. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Unified Multimodal Models (UMMs) often rely on separate visual representations for understanding and generation, increasing visual context length and complicating integration with established vision-language pretraining pipelines. Recent advances in pixel-space modeling offer an encoder-free alternative, but extending this paradigm from images to videos is non-trivial: video understanding and generation adopt different temporal representations, leaving the design of a unified visual interface an open question. We present PixelUMM, an encoder-free model for unified image and video understanding and generation directly in pixel space. PixelUMM represents images as spatial patches and videos as spatiotemporal tubelets, connecting raw pixels to a shared multimodal backbone through single-layer linear projections. Its Mixture-of-Transformers architecture combines shared attention with task-specific parameters and extends clean-pixel prediction to video generation, jointly supporting autoregressive text prediction and pixel-space flow matching. Experiments show that PixelUMM achieves competitive performance across image and video understanding and generation tasks. We further conduct empirical studies of key design choices, including decoder design and spatial-temporal patch size, providing insights for future pixel-space unified multimodal models. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-10-01 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.38597 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T14:19:00.583Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).