Metraipolis › archive › dc_ef040cf410cf

Topology-Informed Visual Prompting For Vision Language Action Policies — arXiv:2609.23944

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.23944
License (as archived): CC0-1.0
Archived: 2026-10-09T22:24:10.309Z
SHA-256 of the archived text: 79e74038f520a360d9cb5a5d861564f2b71a16dfa9ad82355f52cf7a15006d55
Stable URL of this page: https://metraipolis.com/documento/dc_ef040cf410cf

How to cite

Topology-Informed Visual Prompting For Vision Language Action Policies — arXiv:2609.23944 — arXiv (en), https://arxiv.org/abs/2609.23944 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ef040cf410cf (Metraipolis archive dc_ef040cf410cf, sha256 79e74038f520a360d9cb5a5d861564f2b71a16dfa9ad82355f52cf7a15006d55)

BibTeX

@misc{dcef040cf410cf,
  title = {Topology-Informed Visual Prompting For Vision Language Action Policies — arXiv:2609.23944},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.23944},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ef040cf410cf; sha256 79e74038f520a360d9cb5a5d861564f2b71a16dfa9ad82355f52cf7a15006d55},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.23944. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Vision-language-action (VLA) policies can struggle with manipulation tasks with complex obstacle geometries due to partial observability. These complex geometries can lead to similar visual observations or robot configurations requiring qualitatively different actions, a distinction that can be quantified using topological signatures. While motion planners with full knowledge of environment geometries and object states can reason about these signatures in planning, this information is often not known at deployment. To address this issue, we present a topology-guided visual-prompting framework that uses simulation-based planning to augment a nominal demonstration dataset and provides vision-based guidance at deployment. Our method uses a Gauss-Linking-Integral topological signature representation to capture important topological properties of the environment. Using privileged geometry information from a simulation approximation of our environment, we augment a VLA fine-tuning dataset with trajectories that move the system to a demonstrated signature and, from the new configuration, resume task execution. A vision-language model (VLM) is fine-tuned on the same dataset to both predict signatures from live camera observations and predict end-effector waypoints, which are rendered as visual prompts on the observations to guide the VLA. Across three simulated bimanual tasks and a real-world box pickup task, our method outperforms a VLA fine-tuned only on nominal demonstrations and a VLM-prompting baseline that can remove topology-relevant information from observations. On hardware, it exceeds the strongest baseline by 40% in task success. Project website: this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-22 in cs.RO).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2609.23944 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T21:59:59.651Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).