Metraipolis › archive › dc_eb02725966f6

When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models — arXiv:2609.39971

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.39971
License (as archived)
CC0-1.0
Archived
2026-10-07T13:47:48.955Z
SHA-256 of the archived text
d28ae2d395f4fefb67538d29a00e13dececbfcaac09ef4421f7029db9c4b2d95
Stable URL of this page
https://metraipolis.com/documento/dc_eb02725966f6

How to cite

When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models — arXiv:2609.39971 — arXiv (en), https://arxiv.org/abs/2609.39971 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_eb02725966f6 (Metraipolis archive dc_eb02725966f6, sha256 d28ae2d395f4fefb67538d29a00e13dececbfcaac09ef4421f7029db9c4b2d95)
BibTeX
@misc{dceb02725966f6,
  title = {When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models — arXiv:2609.39971},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.39971},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_eb02725966f6; sha256 d28ae2d395f4fefb67538d29a00e13dececbfcaac09ef4421f7029db9c4b2d95},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39971. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Vision-language-action (VLA) models can exceed 90% success on in-distribution tasks and withstand nuisance changes that preserve the required action, yet fail under counterfactual changes that demand a different action. Aggregate robustness scores can therefore conceal a more specific failure, in which a policy responds to both language and vision yet does not combine them to select the action the task requires. We call this failure instruction-action binding. Instructions cue familiar trajectory families, and visual feedback adjusts their execution. Behavioral analyses of fine-tuned $\pi_{0.5}$ and GR00T-N1.7 policies reveal that failed rollouts often retain the source behavior or switch to another demonstrated task. These switches show that language is not simply ignored. Readouts and interventions connect these choices to task-conditioned internal states. Our analysis of the imitation objective shows how narrow conditional action support can leave grounded and instruction-keyed solutions indistinguishable on the demonstrations. This motivates Equivariant Counterfactual Training (ECT), which acts at two levels. ECT data supply valid demonstrations in which the same instruction requires different actions in distinguishable scenes, while the ECT loss trains each demonstration with its counterpart in the same update. In a controlled LIBERO-PRO comparison, full ECT raises $\pi_{0.5}$'s mean position-swap success from 36% to 59%. On CALVIN, where counterparts already occur in the original data, the ECT loss improves five-task completion without new demonstrations. On a real UR5e under a fixed demonstration budget, full ECT raises unseen-position success from 8% to 88%.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.LG).
Categorie arXiv: Robotics (cs.RO) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.39971 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T13:18:10.681Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).