Metraipolis › archive › dc_b98de2f5fbf5

RACE: Calibrated Cumulative Scoring for Rollout-Free Robot Policy Checkpoint Selection — arXiv:2611.05838

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.05838
License (as archived)
CC0-1.0
Archived
2026-11-05T18:17:35.832Z
SHA-256 of the archived text
f9a85119c033e7f53e9090b7c8fae7141e9b5b78e05e412ca4c60ee24113fc07
Stable URL of this page
https://metraipolis.com/documento/dc_b98de2f5fbf5

How to cite

RACE: Calibrated Cumulative Scoring for Rollout-Free Robot Policy Checkpoint Selection — arXiv:2611.05838 — arXiv (en), https://arxiv.org/abs/2611.05838 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b98de2f5fbf5 (Metraipolis archive dc_b98de2f5fbf5, sha256 f9a85119c033e7f53e9090b7c8fae7141e9b5b78e05e412ca4c60ee24113fc07)
BibTeX
@misc{dcb98de2f5fbf5,
  title = {RACE: Calibrated Cumulative Scoring for Rollout-Free Robot Policy Checkpoint Selection — arXiv:2611.05838},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05838},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b98de2f5fbf5; sha256 f9a85119c033e7f53e9090b7c8fae7141e9b5b78e05e412ca4c60ee24113fc07},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05838. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Assessing the deployment performance of learned robot policies during training is challenging, as directly measuring task success requires costly real-world trials for each candidate checkpoint. Averaged validation loss offers a rollout-free alternative, but its checkpoint rankings can be inconsistent with deployment performance. We analyze two limitations of this criterion: it penalizes action deviations without considering whether the policy can recover from them, and gives later reference predictions equal weight even when earlier deviations may lead execution away from the reference. We propose RACE, a novel rollout-free metric for ranking checkpoints using a fixed set of held-out successful trajectories. RACE calibrates action deviations against state-conditional thresholds derived from action differences at similar states in these trajectories. We aggregate these scores through a running product, so that a substantial deviation reduces the contribution of subsequent reference predictions. Averaging the resulting cumulative scores over time and across trajectories yields the checkpoint score. In simulation, RACE outperforms baselines across validation sources, while several baselines degrade sharply when the validation set changes. On real robots, its rankings align with deployment success across the evaluated policy backbones and tasks of varying difficulty. Perturbation analyses further show that RACE distinguishes deviations with markedly different effects on task success, even when baselines respond similarly.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.05838 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-05T18:12:49.037Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).