Metraipolis › archive › dc_b917ae6f89ea

Beyond Single-Axis Testing: Paired Evaluation of Compound Robustness in Vision-Language-Action Policies — arXiv:2609.15940

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.15940
License (as archived)
CC0-1.0
Archived
2026-10-16T02:14:56.439Z
SHA-256 of the archived text
e8b1a9d7c3a36d99844b72325b190e03df919bd61128bd5a6cc45dcc6ee6e6da
Stable URL of this page
https://metraipolis.com/documento/dc_b917ae6f89ea

How to cite

Beyond Single-Axis Testing: Paired Evaluation of Compound Robustness in Vision-Language-Action Policies — arXiv:2609.15940 — arXiv (en), https://arxiv.org/abs/2609.15940 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b917ae6f89ea (Metraipolis archive dc_b917ae6f89ea, sha256 e8b1a9d7c3a36d99844b72325b190e03df919bd61128bd5a6cc45dcc6ee6e6da)
BibTeX
@misc{dcb917ae6f89ea,
  title = {Beyond Single-Axis Testing: Paired Evaluation of Compound Robustness in Vision-Language-Action Policies — arXiv:2609.15940},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.15940},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b917ae6f89ea; sha256 e8b1a9d7c3a36d99844b72325b190e03df919bd61128bd5a6cc45dcc6ee6e6da},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.15940. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Vision-language-action policies are typically evaluated one perturbation at a time, providing a useful diagnosis of their sensitivity to individual distribution shifts. Real-world deployment, however, may involve several shifts simultaneously, and it remains unclear how these individual robustness measurements compose. We ask whether compound robustness can be inferred from single-axis evaluations. We introduce LIBERO-CTRL, a six-axis benchmark that pairs each initial state across single-axis conditions and a matched simultaneous condition. This design reveals two opposing outcome changes that aggregate success rates cannot distinguish: emergent failures, where all single-axis rollouts succeed but the simultaneous rollout fails, and compensated successes, where at least one single-axis rollout fails but the simultaneous rollout succeeds. Because one transition decreases compound success while the other increases it, they can cancel, making aggregate compound performance appear consistent with single-axis measurements even when individual outcomes differ substantially. These opposing transitions can largely cancel in aggregate: even when the difference between the two transition rates is not statistically distinguishable from zero, as many as 29.0% of matched initial states still change outcome. Across six policies and three severity levels, such outcome changes reach 34.5% in the most affected condition. The relative prevalence of the two transitions varies across policies and severities, while the transition rates remain similar under independent re-evaluation of stochastic policies. Compound robustness therefore cannot be characterized from aggregate single-axis success rates alone; matched per-instance evaluation is needed to reveal how joint perturbations alter behavior.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-15 in cs.RO).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2609.15940 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T01:52:32.399Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).