Metraipolis › archive › dc_63c6fe7ef62a
LAWM-3D: Learning 3D-Aware Latent Actions from Human Videos for Generalizable Robot World Models — arXiv:2608.05706
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.05706
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T10:36:07.942Z
- SHA-256 of the archived text
aed06e52820315ca61e717b010d14b788ae0e6f2efcb4c9c7bf108467e7b9b8c- Stable URL of this page
https://metraipolis.com/documento/dc_63c6fe7ef62a
How to cite
LAWM-3D: Learning 3D-Aware Latent Actions from Human Videos for Generalizable Robot World Models — arXiv:2608.05706 — arXiv (en), https://arxiv.org/abs/2608.05706 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_63c6fe7ef62a (Metraipolis archive dc_63c6fe7ef62a, sha256 aed06e52820315ca61e717b010d14b788ae0e6f2efcb4c9c7bf108467e7b9b8c)
BibTeX
@misc{dc63c6fe7ef62a,
title = {LAWM-3D: Learning 3D-Aware Latent Actions from Human Videos for Generalizable Robot World Models — arXiv:2608.05706},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.05706},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_63c6fe7ef62a; sha256 aed06e52820315ca61e717b010d14b788ae0e6f2efcb4c9c7bf108467e7b9b8c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.05706. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
World models enable agents to perform forward rollout and planning without real-world interaction. However, their application in open-world embodied intelligence remains limited by the high cost of action annotations and the heterogeneity of action spaces across platforms. Recently, latent action models (LAMs) have alleviated this bottleneck by learning action representations directly from unlabeled human videos in a self-supervised manner. Nevertheless, most existing LAMs rely on single-view inputs and operate primarily in 2D pixel space, raising a fundamental question: can simply incorporating multi-view videos into LAM training endow the learned latent actions with 3D-aware perception? Our study shows that the answer is negative. The primary reasons lie in future-frame appearance leakage as well as inter-camera appearance discrepancies and viewpoint variations. To address these issues, we propose LAWM-3D, which introduces three tightly coupled key designs: (1) a multi-view invariant unified action tokenization scheme for learning 3D-aware latent actions; (2) a geometric alignment constraint that anchors intermediate encoder features to a pretrained 3D foundation model, thereby explicitly providing cross-view geometric correspondences; and (3) a non-injective RGB-D joint reconstruction objective that prevents shortcut learning from future-frame appearance information, forcing the LAM to focus supervision on motion cues with geometric significance. Importantly, these components are not simply stacked but are tightly coupled through a unified motivation. Built upon a two-stage paradigm of large-scale human video pretraining followed by robot fine-tuning, extensive experiments demonstrate that the proposed 3D-aware latent actions significantly improve world model performance, achieving SOTA results in generation quality, physical consistency, and generalization ability. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-07 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2608.05706 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T10:23:31.001Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).