Metraipolis › archive › dc_d84b6911ff75

Shades of Success: Evaluating Manipulation Quality from Object-Centric Constraints — arXiv:2611.01250

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01250
License (as archived)
CC0-1.0
Archived
2026-11-03T18:57:08.153Z
SHA-256 of the archived text
a69f48cdeb6bfd4ed55e6aa2c0cf2c151eafaaab7d1cf2cbb72eea26e8787f3a
Stable URL of this page
https://metraipolis.com/documento/dc_d84b6911ff75

How to cite

Shades of Success: Evaluating Manipulation Quality from Object-Centric Constraints — arXiv:2611.01250 — arXiv (en), https://arxiv.org/abs/2611.01250 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d84b6911ff75 (Metraipolis archive dc_d84b6911ff75, sha256 a69f48cdeb6bfd4ed55e6aa2c0cf2c151eafaaab7d1cf2cbb72eea26e8787f3a)
BibTeX
@misc{dcd84b6911ff75,
  title = {Shades of Success: Evaluating Manipulation Quality from Object-Centric Constraints — arXiv:2611.01250},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01250},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d84b6911ff75; sha256 a69f48cdeb6bfd4ed55e6aa2c0cf2c151eafaaab7d1cf2cbb72eea26e8787f3a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01250. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As learned manipulation policies become increasingly capable of solving practical manipulation tasks, assessing how well a task is performed becomes a fundamental challenge. While binary labels confirm task success, they cannot distinguish successful executions with different levels of geometric precision. To address this, we introduce SPECTRA, an object-centric framework for quantifying execution quality in manipulation of rigid objects. From a small set of oracle demonstrations generated in simulation, SPECTRA represents object interactions through progress-aligned relative poses. SPECTRA identifies which relative-pose directions are constrained and their admissible variations from demonstrations. These constraints establish a reference for evaluating robotic manipulation tasks. To score an unseen execution, deviations beyond the admissible variation are summarized by a Task Quality Index Q in [0,1]. Across seven simulated tasks, SPECTRA recovers constraints with 94% accuracy using 30 demonstrations per task. On rollouts from three learned policies, its scores achieve pooled Spearman correlations with human judgments of p=0.81 over all rollouts and p=0.61 among successful rollouts, exceeding all baselines in both subsets. Using the task reference learned in simulation, we use SPECTRA to select the highest-scoring demonstrations for training a diffusion policy on hardware. Exploratory results show higher success rate than training on randomly selected demonstrations.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.RO, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2611.01250 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:17:07.728Z dalla pagina https://arxiv.org/list/cs.RO/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).