Metraipolis › archive › dc_5dfda746d2d6
Behavior-Skill: A Fine-Grained Benchmark for Evaluating Vision-Language-Action Policies in Long-Horizon Tasks — arXiv:2608.30536
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.30536
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T10:06:58.116Z
- SHA-256 of the archived text
1d79e2df0c46c68e0de01526c67651005e02a2e1a3882d1151532091c26fafe0- Stable URL of this page
https://metraipolis.com/documento/dc_5dfda746d2d6
How to cite
Behavior-Skill: A Fine-Grained Benchmark for Evaluating Vision-Language-Action Policies in Long-Horizon Tasks — arXiv:2608.30536 — arXiv (en), https://arxiv.org/abs/2608.30536 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5dfda746d2d6 (Metraipolis archive dc_5dfda746d2d6, sha256 1d79e2df0c46c68e0de01526c67651005e02a2e1a3882d1151532091c26fafe0)
BibTeX
@misc{dc5dfda746d2d6,
title = {Behavior-Skill: A Fine-Grained Benchmark for Evaluating Vision-Language-Action Policies in Long-Horizon Tasks — arXiv:2608.30536},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.30536},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5dfda746d2d6; sha256 1d79e2df0c46c68e0de01526c67651005e02a2e1a3882d1151532091c26fafe0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.30536. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reliable execution of long-horizon mobile manipulation tasks remains challenging because overall task success depends on the successful completion of multiple constituent skills. Existing benchmarks, however, still rely primarily on full-task rollouts and aggregate task-level metrics, making intermediate failures difficult to observe and analyze. We present Behavior-Skill, a benchmark that reformulates the learning and evaluation of long-horizon tasks around executable constituent skills. It contains 235,492 skill instances from 10,000 demonstrations across 50 household tasks and 34 semantic skill categories. Each instance pairs a skill instruction with an aligned observation-action segment, and is further associated with a restorable intermediate state and a skill success condition to enable independent evaluation under valid preconditions. We further introduce trajectory-level and skill-level metrics to characterize policy capability beyond aggregate task success. Extensive experiments across representative VLA policies including pi0.5 and GR00T on the complete 50-task benchmark show that failures are highly non-uniform across skills, with contact-rich manipulation skills forming persistent bottlenecks. These results demonstrate that Behavior-Skill complements full-task evaluation by exposing intermediate capability profiles for analyzing and improving long-horizon VLA policies. Behavior-Skill is publicly available at this https URL . Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-01 in cs.RO). Categorie arXiv: Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2608.30536 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T09:58:57.817Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).