Metraipolis › archive › dc_5dfda746d2d6

Behavior-Skill: A Fine-Grained Benchmark for Evaluating Vision-Language-Action Policies in Long-Horizon Tasks — arXiv:2608.30536

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.30536
License (as archived)
CC0-1.0
Archived
2026-10-17T10:06:58.116Z
SHA-256 of the archived text
1d79e2df0c46c68e0de01526c67651005e02a2e1a3882d1151532091c26fafe0
Stable URL of this page
https://metraipolis.com/documento/dc_5dfda746d2d6

How to cite

Behavior-Skill: A Fine-Grained Benchmark for Evaluating Vision-Language-Action Policies in Long-Horizon Tasks — arXiv:2608.30536 — arXiv (en), https://arxiv.org/abs/2608.30536 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5dfda746d2d6 (Metraipolis archive dc_5dfda746d2d6, sha256 1d79e2df0c46c68e0de01526c67651005e02a2e1a3882d1151532091c26fafe0)
BibTeX
@misc{dc5dfda746d2d6,
  title = {Behavior-Skill: A Fine-Grained Benchmark for Evaluating Vision-Language-Action Policies in Long-Horizon Tasks — arXiv:2608.30536},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.30536},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5dfda746d2d6; sha256 1d79e2df0c46c68e0de01526c67651005e02a2e1a3882d1151532091c26fafe0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.30536. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reliable execution of long-horizon mobile manipulation tasks remains challenging because overall task success depends on the successful completion of multiple constituent skills. Existing benchmarks, however, still rely primarily on full-task rollouts and aggregate task-level metrics, making intermediate failures difficult to observe and analyze. We present Behavior-Skill, a benchmark that reformulates the learning and evaluation of long-horizon tasks around executable constituent skills. It contains 235,492 skill instances from 10,000 demonstrations across 50 household tasks and 34 semantic skill categories. Each instance pairs a skill instruction with an aligned observation-action segment, and is further associated with a restorable intermediate state and a skill success condition to enable independent evaluation under valid preconditions. We further introduce trajectory-level and skill-level metrics to characterize policy capability beyond aggregate task success. Extensive experiments across representative VLA policies including pi0.5 and GR00T on the complete 50-task benchmark show that failures are highly non-uniform across skills, with contact-rich manipulation skills forming persistent bottlenecks. These results demonstrate that Behavior-Skill complements full-task evaluation by exposing intermediate capability profiles for analyzing and improving long-horizon VLA policies. Behavior-Skill is publicly available at this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-01 in cs.RO).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2608.30536 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T09:58:57.817Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).