Metraipolis › archive › dc_ed5b35dc38ef

WorldFailBench: Do World Models Understand Policy Failures? — arXiv:2610.33565

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.33565
License (as archived)
CC0-1.0
Archived
2026-10-27T08:00:20.106Z
SHA-256 of the archived text
e6b371be5772635ecbbec6a7b73c1b452ca696bf076b3d06aa0f1a128d587e84
Stable URL of this page
https://metraipolis.com/documento/dc_ed5b35dc38ef

How to cite

WorldFailBench: Do World Models Understand Policy Failures? — arXiv:2610.33565 — arXiv (en), https://arxiv.org/abs/2610.33565 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ed5b35dc38ef (Metraipolis archive dc_ed5b35dc38ef, sha256 e6b371be5772635ecbbec6a7b73c1b452ca696bf076b3d06aa0f1a128d587e84)
BibTeX
@misc{dced5b35dc38ef,
  title = {WorldFailBench: Do World Models Understand Policy Failures? — arXiv:2610.33565},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.33565},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ed5b35dc38ef; sha256 e6b371be5772635ecbbec6a7b73c1b452ca696bf076b3d06aa0f1a128d587e84},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.33565. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Robot policy evaluation informs deployment decisions and policy improvement, but physical testing and simulator preparation require substantial effort. World models offer a complementary route through closed-loop interaction in generated environments. Recent benchmarks have demonstrated their potential as policy evaluators through outcome-based evaluation (i.e., assessing task success or failure). However, such evaluation does not establish whether world models faithfully simulate how policies fail. Misrepresenting failures can lead to incorrect diagnoses of policy weaknesses and make policies appear ready for deployment even when small scene changes cause them to fail. We therefore ask: do world models faithfully simulate how and at which stages policies fail, and under what conditions they transition from success to failure? To answer this question, we introduce WorldFailBench with three evaluation tasks. Failure reason tests whether models reproduce the observable errors policies make. Failure stage assesses whether failures concentrate at the same stages of full-task execution. Failure boundary examines whether models reproduce the conditions under which policies transition from success to failure under controlled scene changes in simulation. The benchmark covers manipulation tasks in the LIBERO and RoboTwin simulation environments and real-world manipulation tasks performed by the G1 humanoid robot. Experiments across seven world models reveal substantial gaps in their understanding of policy failures, providing guidance for developing more reliable world models for policy evaluation.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.33565 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:55:30.834Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).