Metraipolis › archive › dc_b571c568fae2
DreamWAM: Beyond RGB Future Prediction for World Action Models — arXiv:2608.04996
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.04996
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T11:50:05.903Z
- SHA-256 of the archived text
ca884897ba5fe7458a62df389ed2c92ae1d65ac2270cfa37c10d9740e242debc- Stable URL of this page
https://metraipolis.com/documento/dc_b571c568fae2
How to cite
DreamWAM: Beyond RGB Future Prediction for World Action Models — arXiv:2608.04996 — arXiv (en), https://arxiv.org/abs/2608.04996 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b571c568fae2 (Metraipolis archive dc_b571c568fae2, sha256 ca884897ba5fe7458a62df389ed2c92ae1d65ac2270cfa37c10d9740e242debc)
BibTeX
@misc{dcb571c568fae2,
title = {DreamWAM: Beyond RGB Future Prediction for World Action Models — arXiv:2608.04996},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.04996},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b571c568fae2; sha256 ca884897ba5fe7458a62df389ed2c92ae1d65ac2270cfa37c10d9740e242debc},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04996. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
World Action Models (WAMs) learn action-relevant representations by predicting how the observed world will evolve. Most existing WAMs define this future in RGB space, where task-relevant state transitions are entangled with nuisance variations in texture, illumination, background, and viewpoint. We argue that WAMs should explicitly predict action-relevant future state rather than relying on RGB prediction alone. We introduce DreamWAM, which reformulates future prediction as structured world modeling beyond RGB, representing future states through complementary views of appearance, motion, geometry, and semantics. During training, DreamWAM combines joint latent denoising of RGB and motion with lightweight gated residual branches for geometry and semantics. Shared attention between VideoDiT and ActionDiT allows the action branch to learn from these future-state predictions, while all beyond-RGB supervision branches are disabled at inference and deployment remains RGB-only. Across both no-rollout and joint video-action inference, DreamWAM consistently improves the matched RGB-only baselines on LIBERO, from 97.30\% to 98.40\% and from 98.00\% to 98.90\%, respectively. The gains become larger under unseen LIBERO-Plus perturbations, from 51.36\% to 63.44\% and from 69.16\% to 75.47\%. The same robustness extends to real-world manipulation, where DreamWAM attains an average success rate of 74.4\% across unseen changes in lighting, background, and object layout, compared with 55.6\% for Fast-WAM-Joint. These results show that robust world-action learning depends not only on predicting the future, but on representing it in a form that matters for action. The code and models are publicly released at this https URL . Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-06 in cs.RO). Categorie arXiv: Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2608.04996 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T11:39:29.948Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).