Metraipolis › archive › dc_fb0137df66fb

Hybrid Particle-Voxel Dynamics for Multimodal 3D World Simulation — arXiv:2611.04492

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.04492
License (as archived)
CC0-1.0
Archived
2026-11-04T12:42:41.479Z
SHA-256 of the archived text
4c7032dd816281be1eff305dc8e2b1bce101a75f3cb45e6505e56b14e9054cb9
Stable URL of this page
https://metraipolis.com/documento/dc_fb0137df66fb

How to cite

Hybrid Particle-Voxel Dynamics for Multimodal 3D World Simulation — arXiv:2611.04492 — arXiv (en), https://arxiv.org/abs/2611.04492 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fb0137df66fb (Metraipolis archive dc_fb0137df66fb, sha256 4c7032dd816281be1eff305dc8e2b1bce101a75f3cb45e6505e56b14e9054cb9)
BibTeX
@misc{dcfb0137df66fb,
  title = {Hybrid Particle-Voxel Dynamics for Multimodal 3D World Simulation — arXiv:2611.04492},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.04492},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fb0137df66fb; sha256 4c7032dd816281be1eff305dc8e2b1bce101a75f3cb45e6505e56b14e9054cb9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04492. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Predicting how the world evolves in response to robot actions is central to policy learning, evaluation, and planning. However, many action-conditioned world models predict future observations in 2D image or video space without explicitly modeling the underlying 3D scene, making spatial consistency across viewpoints and over long-horizon interactions challenging. We introduce 3D World Simulator, an action-conditioned multimodal world modeling framework that jointly predicts appearance, geometry, and tactile observations from a unified 3D representation. Our approach combines a hybrid particle-voxel autoencoder that compresses point clouds into compact 3D voxel latents with an action-conditioned latent dynamics model that predicts their evolution. The shared 3D representation supports geometrically consistent rendering from arbitrary viewpoints and flexible integration of spatially aligned tactile signals. Our experiments demonstrate interactive, long-horizon, multimodal prediction at 50 Hz. We then show how our model enables scalable data generation for policy training and evaluation using tactile and multi-view visual observations, including wrist-mounted and novel camera views. Furthermore, we scale our method to large datasets containing diverse objects and demonstrate model-based planning with unseen objects and robot configurations. Together, these results highlight the potential of hybrid particle-voxel dynamics as a foundation for multimodal 3D world simulation.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.04492 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T12:29:47.392Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).