Metraipolis › archive › dc_89d74de5dbac

What Matters, When? Diagnosing and Improving Conditional Visual Grounding in Visuomotor Imitation Policies — arXiv:2609.05376

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.05376
License (as archived)
CC0-1.0
Archived
2026-10-16T22:29:24.645Z
SHA-256 of the archived text
416cf3eae5d6a9d4c837ea07e8c0bd915cef1b24fb5fa89546f0dd5216f1a6c1
Stable URL of this page
https://metraipolis.com/documento/dc_89d74de5dbac

How to cite

What Matters, When? Diagnosing and Improving Conditional Visual Grounding in Visuomotor Imitation Policies — arXiv:2609.05376 — arXiv (en), https://arxiv.org/abs/2609.05376 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_89d74de5dbac (Metraipolis archive dc_89d74de5dbac, sha256 416cf3eae5d6a9d4c837ea07e8c0bd915cef1b24fb5fa89546f0dd5216f1a6c1)
BibTeX
@misc{dc89d74de5dbac,
  title = {What Matters, When? Diagnosing and Improving Conditional Visual Grounding in Visuomotor Imitation Policies — arXiv:2609.05376},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.05376},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_89d74de5dbac; sha256 416cf3eae5d6a9d4c837ea07e8c0bd915cef1b24fb5fa89546f0dd5216f1a6c1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.05376. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Visuomotor imitation policies can achieve high performance under in-distribution visual conditions yet fail when visually similar objects or receptacles are introduced. We study this behavior as a problem of conditional visual grounding: the visual target required for successful control changes with the manipulation phase and, in more complex tasks, with the observed task state. Using Action Chunking with Transformers (ACT), we systematically introduce distractor objects and receptacles with controlled color and shape similarity and localize failures to picking and placement. We find that distractor sensitivity is specific to both the type of visual similarity and the manipulation stage. Guided by this diagnosis, we evaluate distractor augmentation, phase-dependent attention regularization, and appearance-based visual prompting as complementary interventions for improving target selection while preserving spatial information required for control. These interventions substantially improve robustness in simulation and on a physical UR3e. We further examine the same failure pattern in a pretrained vision-language-action policy on a state-conditioned instrument-handling task, where the observed state of a medical instrument determines the correct destination. Together, the results show that visual distractors can cause incorrect object or destination selection even when the underlying manipulation skill remains intact, and that explicitly improving target selection can substantially recover performance across distinct visuomotor policy-learning regimes.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-07 in cs.AI).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.05376 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T22:18:11.593Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).