Metraipolis › archive › dc_504f8eccce14
Vision-Language Models Are Joint Denoisers in Robotic Action Generation — arXiv:2611.01215
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.01215
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:46:21.348Z
- SHA-256 of the archived text
190566643f4757383d592ffdcac588f054b978fb2801e1f6dd999b43c8fde733- Stable URL of this page
https://metraipolis.com/documento/dc_504f8eccce14
How to cite
Vision-Language Models Are Joint Denoisers in Robotic Action Generation — arXiv:2611.01215 — arXiv (en), https://arxiv.org/abs/2611.01215 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_504f8eccce14 (Metraipolis archive dc_504f8eccce14, sha256 190566643f4757383d592ffdcac588f054b978fb2801e1f6dd999b43c8fde733)
BibTeX
@misc{dc504f8eccce14,
title = {Vision-Language Models Are Joint Denoisers in Robotic Action Generation — arXiv:2611.01215},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.01215},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_504f8eccce14; sha256 190566643f4757383d592ffdcac588f054b978fb2801e1f6dd999b43c8fde733},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01215. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Diffusion-based vision-language-action (VLA) models cascade pretrained vision-language models (VLMs) with diffusion action experts, but typically restrict the VLM to providing static conditioning. This separation limits how much pretrained perception and reasoning can guide action generation, especially in out-of-distribution settings. We introduce JADE, a Joint Action DEnoiser that enables the VLM and action expert to jointly refine actions. Our key insight is that structured action latents allow the VLM to read the current noisy action together with visual observations and language instructions. At every denoising step, the VLM contextualizes the noisy action and guides the action expert through gated cross-attention, so that its guidance evolves with the action being generated. We further supervise the VLM to recover clean action latents from noisy ones, which gives it a direct action-learning signal and couples perception with control. Additionally, joint denoising enables JADE to generate actions with fewer sampling steps, helping offset the additional cost of VLM participation. An adaptive step-size schedule further balances task performance and inference latency. On RoboDojo, JADE achieves an average success rate of 12.67%, the highest among the evaluated $\pi_{0.5}$-based methods, and is competitive with recent VLA and WAM foundation models pretrained at scale. It also improves performance on RoboTwin 2.0 and raises the average success rate across four real-world bimanual tasks by 20.8 percentage points compared with $\pi_{0.5}$.
Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.01215 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:39:50.457Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).