Metraipolis › archive › dc_673a2babb95d

DepthWorld: 3D World Model for Robot Manipulation — arXiv:2610.08780

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.08780
License (as archived)
CC0-1.0
Archived
2026-10-07T11:22:11.236Z
SHA-256 of the archived text
05db27055858178a4c7d76cb9b30e0fec30b997506557e5b3dc3520e6968cb30
Stable URL of this page
https://metraipolis.com/documento/dc_673a2babb95d

How to cite

DepthWorld: 3D World Model for Robot Manipulation — arXiv:2610.08780 — arXiv (en), https://arxiv.org/abs/2610.08780 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_673a2babb95d (Metraipolis archive dc_673a2babb95d, sha256 05db27055858178a4c7d76cb9b30e0fec30b997506557e5b3dc3520e6968cb30)
BibTeX
@misc{dc673a2babb95d,
  title = {DepthWorld: 3D World Model for Robot Manipulation — arXiv:2610.08780},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.08780},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_673a2babb95d; sha256 05db27055858178a4c7d76cb9b30e0fec30b997506557e5b3dc3520e6968cb30},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08780. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

World models offer a data-driven alternative to traditional simulators for robotics, with applications spanning policy evaluation, improvement, and planning. All of these uses depend on faithful 3D geometry, yet current video-based world models are trained on RGB alone and produce rollouts that look correct frame-by-frame but do not compose into a consistent 3D world. Closing this gap requires progress on two fronts: large-scale 3D supervision for manipulation, and an architecture that can absorb it without disturbing strong pretrained video priors. We introduce a calibration pipeline that combines learned stereo depth with a joint factor graph, pooling all episodes collected from the same physical robot to recover its shared kinematic parameters alongside per-scene extrinsics. Applied to the DROID dataset, this yields DROID-3D, a calibrated 3D dataset providing dense metric depth and recalibrated multi-view extrinsics (achieving <0.7 px reprojection error on 90% of episodes for external cameras). We then train DepthWorld, a Stable Video Diffusion-based world model that jointly predicts multi-view RGB and depth via spatial latent tiling, leaving the pretrained Variational Autoencoder (VAE) unchanged. Depth supervision improves RGB prediction itself by +1.48 dB PSNR over an identical RGB-only baseline at equal training budget, while simultaneously yielding accurate metric depth for downstream geometric reasoning.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2610.08780 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:15:24.866Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).