Metraipolis › archive › dc_e4b233d05018

LD4WAM: Learning Latent Dynamics from Human Videos for World Action Models — arXiv:2608.22403

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.22403
License (as archived)
CC0-1.0
Archived
2026-10-18T09:02:04.674Z
SHA-256 of the archived text
1b5fdf818775df7a4e09b25be1981f42af96c17e51ac168e6a99e012b4b49ca6
Stable URL of this page
https://metraipolis.com/documento/dc_e4b233d05018

How to cite

LD4WAM: Learning Latent Dynamics from Human Videos for World Action Models — arXiv:2608.22403 — arXiv (en), https://arxiv.org/abs/2608.22403 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e4b233d05018 (Metraipolis archive dc_e4b233d05018, sha256 1b5fdf818775df7a4e09b25be1981f42af96c17e51ac168e6a99e012b4b49ca6)
BibTeX
@misc{dce4b233d05018,
  title = {LD4WAM: Learning Latent Dynamics from Human Videos for World Action Models — arXiv:2608.22403},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.22403},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e4b233d05018; sha256 1b5fdf818775df7a4e09b25be1981f42af96c17e51ac168e6a99e012b4b49ca6},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.22403. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Human video is playing an increasingly central role in training World Action Models (WAMs), owing to its diversity and low collection cost relative to teleoperated robot data. However, most WAMs learn from such video only by predicting pixel-level future frames, giving dynamics that are not directly actionable, whereas motion retargeting recovers directly actionable actions but leaves a large visual gap across embodiments. We therefore propose motion-aligned latent dynamics as an embodiment-agnostic representation to bridge video priors and low-level actions. We further present LD4WAM, which pairs a Latent Dynamics Model trained with semantic reconstruction and real motion alignment with a World Dynamics Action Model built as a mixture-of-transformers (MoT), which preserves full future-video generation and uses learnable queries to distill these latent dynamics from generated futures for action conditioning. Pretrained on our curated unified dataset of over 5{,}000 hours of human and robot data, LD4WAM performs strongly in RoboTwin simulation and on real robots equipped with both grippers and dexterous hands, while generalizing well to unseen objects and backgrounds.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-25 in cs.RO).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2608.22403 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T08:57:47.859Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).