Metraipolis › archive › dc_453d8fa6955f
Dream4ACT: A Shared Visual Action Interface for Multi-Embodiment Video-Action Modeling — arXiv:2609.40153
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.40153
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T15:12:20.510Z
- SHA-256 of the archived text
a2920edcd4caf4a44f51fe1cea096c3f71184d44fa779d30eb0c92c7d1d1bf32- Stable URL of this page
https://metraipolis.com/documento/dc_453d8fa6955f
How to cite
Dream4ACT: A Shared Visual Action Interface for Multi-Embodiment Video-Action Modeling — arXiv:2609.40153 — arXiv (en), https://arxiv.org/abs/2609.40153 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_453d8fa6955f (Metraipolis archive dc_453d8fa6955f, sha256 a2920edcd4caf4a44f51fe1cea096c3f71184d44fa779d30eb0c92c7d1d1bf32)
BibTeX
@misc{dc453d8fa6955f,
title = {Dream4ACT: A Shared Visual Action Interface for Multi-Embodiment Video-Action Modeling — arXiv:2609.40153},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.40153},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_453d8fa6955f; sha256 a2920edcd4caf4a44f51fe1cea096c3f71184d44fa779d30eb0c92c7d1d1bf32},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.40153. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Video generation models (VGMs) offer strong spatiotemporal priors for embodied observation--action modeling. However, joint-space action vectors lack explicit image-space structure and vary in dimensionality and semantics across embodiments, making it challenging to directly leverage the rich spatiotemporal priors of VGMs. End-effector visualizations provide an alternative but do not specify the full articulated configuration needed for robot execution. We present Dream4ACT, a world model built for joint video-action modeling across embodiments. To unify action representations across embodiments, we introduce a shared visual action interface, called action views, which render target joint configurations from four prescribed virtual cameras using URDF-based forward kinematics. This shared visual representation preserves embodiment-specific articulated geometry while allowing observation and action sequences to share a video autoencoder and diffusion transformer. Through masked flow-matching, our model supports forward dynamics, inverse dynamics, and joint observation--action generation within a single jointly trained model by varying which future sequences are corrupted. To recover executable action sequences from predicted action views, we propose a training-free, URDF-constrained multiview recovery mechanism, without a learned embodiment-specific decoder. Dream4ACT achieves an average success rate of 88.98\% on RoboTwin~2.0 and an overall score of 65.66 on TriWorldBench, supporting effective closed-loop manipulation and competitive action-conditioned multiview prediction through the visual action interface. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-10-01 in cs.RO). Categorie arXiv: Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2609.40153 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T14:44:47.492Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).