Metraipolis › archive › dc_738df6069c46

Does Latent Planning Survive Point Clouds? Action-Conditioned JEPA World Models for Geometric Observations and Goals — arXiv:2608.29434

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.29434
License (as archived)
CC0-1.0
Archived
2026-10-17T07:47:44.729Z
SHA-256 of the archived text
83dea19e648742455497bac9feb4cc2af6e5eadbe044d4e9c2d2fea0384d4890
Stable URL of this page
https://metraipolis.com/documento/dc_738df6069c46

How to cite

Does Latent Planning Survive Point Clouds? Action-Conditioned JEPA World Models for Geometric Observations and Goals — arXiv:2608.29434 — arXiv (en), https://arxiv.org/abs/2608.29434 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_738df6069c46 (Metraipolis archive dc_738df6069c46, sha256 83dea19e648742455497bac9feb4cc2af6e5eadbe044d4e9c2d2fea0384d4890)
BibTeX
@misc{dc738df6069c46,
  title = {Does Latent Planning Survive Point Clouds? Action-Conditioned JEPA World Models for Geometric Observations and Goals — arXiv:2608.29434},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.29434},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_738df6069c46; sha256 83dea19e648742455497bac9feb4cc2af6e5eadbe044d4e9c2d2fea0384d4890},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29434. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Latent action world models let agents plan new behaviors at test time by predicting how actions change the environment, and joint-embedding predictive architectures (JEPAs) do so by forecasting future latent states rather than pixels. Yet nearly all such models see the world through a camera, even though robotic manipulation is fundamentally geometric: in robotics goals for manipulation are traditionally specified by target object poses, not by images of the object once placed. We ask whether latent planning survives a shift from appearance to geometry, on the observation side as well as on the goal specifications side. To answer this, we extend the stable-worldmodel evaluation platform with simulated LiDAR-style raycast point clouds as a new sensor modality, and adapt three JEPA designs to point clouds: a frozen-encoder model built on Utonia features, a distribution-prior model based on LeWM, and an action-sensitive model based on Delta-JEPA. We further introduce a goal-encoding mechanism that constructs the goal latent from the current latent and a 3D target pose, removing the need for goal images or goal point clouds. A comparative evaluation of the different anti-collapse mechanisms shows that point-cloud world models can match their image-based counterparts, demonstrating that the modality shift from appearance to geometry is achievable. All models are released as open weights with open-source training and inference code, to make world-model planning accessible for LiDAR-driven and pose-directed robotic tasks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.29434 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:28:26.758Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).