Metraipolis › archive › dc_3c561d70804e

SG-WAM: Text-Grounded and Spatial-aware Semantic Guidance for World-Action Models — arXiv:2608.08839

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.08839
License (as archived)
CC0-1.0
Archived
2026-10-19T09:12:38.488Z
SHA-256 of the archived text
fedf95ee91f5d73da06e2c38ca068d539e54b376bf6b2209d15f4c510573d8b5
Stable URL of this page
https://metraipolis.com/documento/dc_3c561d70804e

How to cite

SG-WAM: Text-Grounded and Spatial-aware Semantic Guidance for World-Action Models — arXiv:2608.08839 — arXiv (en), https://arxiv.org/abs/2608.08839 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3c561d70804e (Metraipolis archive dc_3c561d70804e, sha256 fedf95ee91f5d73da06e2c38ca068d539e54b376bf6b2209d15f4c510573d8b5)
BibTeX
@misc{dc3c561d70804e,
  title = {SG-WAM: Text-Grounded and Spatial-aware Semantic Guidance for World-Action Models — arXiv:2608.08839},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.08839},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3c561d70804e; sha256 fedf95ee91f5d73da06e2c38ca068d539e54b376bf6b2209d15f4c510573d8b5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08839. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

World-Action Models (WAMs) have emerged as a promising paradigm for robotic manipulation. However, most existing WAMs generate future videos and actions by relying mainly on visual cues rather than language instructions, since off-the-shelf text encoders embed instructions independently of visual observations. As a result, the videos predicted by these WAMs are often semantically misaligned with their corresponding language instructions, which degrades the accuracy of the predicted actions. To overcome this limitation, we propose SG-WAM, a semantic guidance method for world-action models that leverages a vision-language model (VLM) as a semantic planner to enhance the instruction-grounding capacity of world-action models. Specifically, we train a VLM-based planner to predict text-grounded and spatial-aware semantic foresight. The text-grounded semantic foresight grounds the instruction by identifying the correct target objects, and the spatial-aware semantic foresight provides the scene geometry for precise manipulation. We then inject this foresight into the world-action model as high-level semantic guidance, ensuring that both future-video generation and action prediction faithfully follow the language instruction. Extensive experiments in simulation and the real world demonstrate the superiority of our semantic guidance method, showcasing precise manipulation and strong instruction-following capabilities.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-11 in cs.RO).
Categorie arXiv: Robotics (cs.RO) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.08839 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T09:11:58.755Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).