Metraipolis › archive › dc_4ac511b79704

Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling — arXiv:2610.39232

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.39232
License (as archived)
CC0-1.0
Archived
2026-11-01T02:41:52.136Z
SHA-256 of the archived text
7d98b516d4eca885e163853e48eb04a62f5cba15e599cb737a1ac0ff69d84f58
Stable URL of this page
https://metraipolis.com/documento/dc_4ac511b79704

How to cite

Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling — arXiv:2610.39232 — arXiv (en), https://arxiv.org/abs/2610.39232 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4ac511b79704 (Metraipolis archive dc_4ac511b79704, sha256 7d98b516d4eca885e163853e48eb04a62f5cba15e599cb737a1ac0ff69d84f58)
BibTeX
@misc{dc4ac511b79704,
  title = {Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling — arXiv:2610.39232},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.39232},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4ac511b79704; sha256 7d98b516d4eca885e163853e48eb04a62f5cba15e599cb737a1ac0ff69d84f58},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.39232. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Embodied world models should not only imagine plausible futures, but also perceive the geometric, semantic, and motion structure of the evolving world. We present GenSight, a unified embodied world model that brings future video generation and dense perception into a single pretrained video diffusion transformer. GenSight uses a unified spatiotemporal representation for video, depth, embodiment segmentation, and first-frame-relative 2D flow. Built on a shared DiT backbone, GenSight uses lightweight task modulation to predict all modalities within the same architecture. All tasks are jointly optimized within a unified velocity prediction space. At inference, GenSight supports two complementary modes: multi-step future video generation with aligned dense predictions and single-step dense perception from an observed video. By unifying modality representation, model architecture, and prediction space, GenSight avoids separate modality-specific pipelines and couples appearance, geometry, semantics, and motion in a shared representation. It promotes cross-modal consistency and allows dense perception to benefit from the spatiotemporal priors learned through video generation. Trained on embodied manipulation data spanning multiple robot embodiments and human-hand interactions, GenSight demonstrates strong performance in both video generation and dense 4D perception, highlighting the effectiveness of unifying seeing and imagining for embodied world modeling.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2610.39232 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-01T02:16:43.914Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).