Metraipolis › archive › dc_3e699e7d52ba

DUET-DINO: Simultaneous Cross-View World Modeling for Latent Planning in Robot Manipulation — arXiv:2609.10506

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.10506
License (as archived)
CC0-1.0
Archived
2026-10-16T11:00:16.489Z
SHA-256 of the archived text
d7ee3a95ec573df935d713dd856d88860476d0b954f9c991a433890f26fc4dac
Stable URL of this page
https://metraipolis.com/documento/dc_3e699e7d52ba

How to cite

DUET-DINO: Simultaneous Cross-View World Modeling for Latent Planning in Robot Manipulation — arXiv:2609.10506 — arXiv (en), https://arxiv.org/abs/2609.10506 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3e699e7d52ba (Metraipolis archive dc_3e699e7d52ba, sha256 d7ee3a95ec573df935d713dd856d88860476d0b954f9c991a433890f26fc4dac)
BibTeX
@misc{dc3e699e7d52ba,
  title = {DUET-DINO: Simultaneous Cross-View World Modeling for Latent Planning in Robot Manipulation — arXiv:2609.10506},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.10506},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3e699e7d52ba; sha256 d7ee3a95ec573df935d713dd856d88860476d0b954f9c991a433890f26fc4dac},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.10506. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Action-conditioned latent world models predict future visual representations, enabling zero-shot goal-conditioned robot planning and control. However, their predictions for fine-grained spatial and rotational actions are unreliable for full 7-DoF end-effector control. To address this gap, we introduce DUET-DINO, a simultaneous cross-view latent world model that jointly learns action-conditioned predictions from static side- and wrist-camera observations through cross-view conditioning. By exploiting complementary global scene and gripper-centric information, DUET-DINO enables latent planning over the full 7-DoF action space. Across spatially diverse reach, orientation-intensive angled-reach, and multi-goal grasp-and-lift tasks, DUET-DINO consistently outperforms single-view and independent dual-view baselines, achieving 92% success on reach, 72.5% on angled-reach, and 60.0% on lift tasks. DUET-DINO is trained from scratch on DROID and RoboArena datasets and generalizes robustly under visual distribution shifts. We further show that while V-JEPA 2 wrist-view predictions underestimate visual dynamics induced by fine-grained actions, DINOv3 predictions better capture action-conditioned scene changes, leading to stronger downstream planning. The code and model checkpoints will be open-sourced. Project page: this https URL

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-10 in cs.CV).
Categorie arXiv: Robotics (cs.RO) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.10506 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T10:02:34.446Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).