Metraipolis › archive › dc_110336b8366b

PRM-as-a-Judge 1.5: A Toolkit for Robot Process Assessment — arXiv:2608.14284

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.14284
License (as archived)
CC0-1.0
Archived
2026-10-19T04:04:38.050Z
SHA-256 of the archived text
41e710b3d29a66be9cf5708d8b8b5e92e9d2e35b5e42971e37ab634e0d233586
Stable URL of this page
https://metraipolis.com/documento/dc_110336b8366b

How to cite

PRM-as-a-Judge 1.5: A Toolkit for Robot Process Assessment — arXiv:2608.14284 — arXiv (en), https://arxiv.org/abs/2608.14284 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_110336b8366b (Metraipolis archive dc_110336b8366b, sha256 41e710b3d29a66be9cf5708d8b8b5e92e9d2e35b5e42971e37ab634e0d233586)
BibTeX
@misc{dc110336b8366b,
  title = {PRM-as-a-Judge 1.5: A Toolkit for Robot Process Assessment — arXiv:2608.14284},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.14284},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_110336b8366b; sha256 41e710b3d29a66be9cf5708d8b8b5e92e9d2e35b5e42971e37ab634e0d233586},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14284. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Fine-grained robotic evaluation matters for understanding embodied models, going beyond binary success rates and rule-based process scores. We present PRM-as-a-Judge 1.5, a toolkit for robot process assessment that turns rollout videos into dense progress curves and derives multiple fine metrics. PRM-as-a-Judge 1.5 introduces three metrics, building on version 1.0, that characterize failure-side progress, post-drawdown recovery, and success-side execution quality, helping users understand embodied model capability. Based on the rollout videos from benchmarks, we perform a comprehensive assessment of the embodied models, providing some fine-grained metric results and key findings. We further introduce RoboPulse++ to evaluate the reliability of process reward models (PRM), providing evaluators with a more accurate testing platform. Moreover, we release a user-friendly assessment suite, including the benchmark, metric implementation, and visualization tools, to support reproducible manipulation process evaluation. We call on the community to rethink how robots are evaluated and establish transparent, procedural, and reproducible assessment as a foundation for the next generation of embodied intelligence.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-17 in cs.CV).
Categorie arXiv: Robotics (cs.RO) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.14284 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T03:57:58.048Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).