Metraipolis › archive › dc_cbdd7634a2bf

Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations — arXiv:2611.03647

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.03647
License (as archived): CC0-1.0
Archived: 2026-11-03T10:38:20.857Z
SHA-256 of the archived text: aab48e08535e1429beed27a777df164d9a47c5d3e2213c04eb2b05858d3a0baa
Stable URL of this page: https://metraipolis.com/documento/dc_cbdd7634a2bf

How to cite

Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations — arXiv:2611.03647 — arXiv (en), https://arxiv.org/abs/2611.03647 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cbdd7634a2bf (Metraipolis archive dc_cbdd7634a2bf, sha256 aab48e08535e1429beed27a777df164d9a47c5d3e2213c04eb2b05858d3a0baa)

BibTeX

@misc{dccbdd7634a2bf,
  title = {Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations — arXiv:2611.03647},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03647},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cbdd7634a2bf; sha256 aab48e08535e1429beed27a777df164d9a47c5d3e2213c04eb2b05858d3a0baa},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03647. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Sensorized human demonstrations offer scalable data for mobile manipulation, but embodiment differences complicate policy learning. We present Pixel-UMI, a pipeline for learning environment-aware whole-body policies from human data. We first introduce Environment-Aware Retargeting (EAR), which combines reconstructed scene geometry with demonstrated head and gripper trajectories to determine robot posture and generate collision-aware joint and base targets. Delay-calibrated action assignment converts these trajectories into actions. However, kinematic differences can leave residual pose errors, creating viewpoint mismatches between human demonstrations and robot observations. We therefore introduce Retargeting-Error-Aware Masking (REAM), which uses these errors during training to reduce reliance on mismatched views at deployment. Finally, because estimated actions cannot fully capture contact-rich interactions, we collect on-policy human corrections through DAgger to improve performance. We evaluate Pixel-UMI on five challenging tasks spanning articulated-object manipulation, deformable-object handling, constrained-space reaching, whole-body collision avoidance, and contact-rich bimanual coordination, achieving 87.0% average task progress. Video results are available at this https URL

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Systems and Control (eess.SY).
Pagina del lavoro: https://arxiv.org/abs/2611.03647 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:29:48.961Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).