Metraipolis › archive › dc_64ff66e84018

ConflictVLA-Bench: Benchmarking Behavioral Responses of Vision-Language-Action Models to Premise Conflicts — arXiv:2609.31792

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.31792
License (as archived)
CC0-1.0
Archived
2026-10-07T20:54:54.455Z
SHA-256 of the archived text
ab886ac931f6eb71d8957ba6b5b08958ed9c292cb87054e4490784ef7c94c371
Stable URL of this page
https://metraipolis.com/documento/dc_64ff66e84018

How to cite

ConflictVLA-Bench: Benchmarking Behavioral Responses of Vision-Language-Action Models to Premise Conflicts — arXiv:2609.31792 — arXiv (en), https://arxiv.org/abs/2609.31792 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_64ff66e84018 (Metraipolis archive dc_64ff66e84018, sha256 ab886ac931f6eb71d8957ba6b5b08958ed9c292cb87054e4490784ef7c94c371)
BibTeX
@misc{dc64ff66e84018,
  title = {ConflictVLA-Bench: Benchmarking Behavioral Responses of Vision-Language-Action Models to Premise Conflicts — arXiv:2609.31792},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31792},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_64ff66e84018; sha256 ab886ac931f6eb71d8957ba6b5b08958ed9c292cb87054e4490784ef7c94c371},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31792. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

While Vision-Language-Action (VLA) models perform strongly on manipulation tasks, their responses to invalid task premises remain underexplored. Existing evaluations of premise conflicts often focus on terminal task outcomes, yet task failure alone cannot distinguish behavioral disengagement from continued pursuit followed by an execution error. We call the latter pattern Failed Persistence. To study this phenomenon, we introduce ConflictVLA-Bench, which pairs conflict rollouts with premise-consistent reference rollouts and evaluates both outcomes and execution processes. Built on LIBERO, the benchmark contains 2,826 prompt-conditioned conflict tasks spanning four conflict families, four structural configurations, and two prompt conditions. Across all eight VLAs, invalid premises reduce original goal completion by at least 17.3 percentage points, with the reduction reaching 56.2 percentage points for OpenVLA. Crucially, even when models succeed on premise-consistent tasks and fail on their matched conflict tasks, they often continue to approach the original targets, retain early trajectory structure, and show limited action magnitude suppression. Failed Persistence therefore recurs across the evaluated models. Explicit premise checking does not consistently produce selective and coordinated behavioral changes. These findings show that terminal failure alone establishes neither behavioral disengagement nor refusal and that outcomes alone are insufficient for VLA evaluation. Experimental data and additional details are available on the project page: this https URL

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2609.31792 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:00.638Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).