Metraipolis › archive › dc_2ea108bbf534

OmniHOI: Dexterous Hand-Object Interaction from Monocular Human Video — arXiv:2610.10855

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.10855
License (as archived): CC0-1.0
Archived: 2026-10-09T06:33:11.064Z
SHA-256 of the archived text: d6ab4a8660dd8de038362a34d6c009a86cbe541af73f5237efcffe9e2046d749
Stable URL of this page: https://metraipolis.com/documento/dc_2ea108bbf534

How to cite

OmniHOI: Dexterous Hand-Object Interaction from Monocular Human Video — arXiv:2610.10855 — arXiv (en), https://arxiv.org/abs/2610.10855 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2ea108bbf534 (Metraipolis archive dc_2ea108bbf534, sha256 d6ab4a8660dd8de038362a34d6c009a86cbe541af73f5237efcffe9e2046d749)

BibTeX

@misc{dc2ea108bbf534,
  title = {OmniHOI: Dexterous Hand-Object Interaction from Monocular Human Video — arXiv:2610.10855},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.10855},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2ea108bbf534; sha256 d6ab4a8660dd8de038362a34d6c009a86cbe541af73f5237efcffe9e2046d749},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.10855. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Monocular videos of human manipulation provide abundant dexterous demonstrations, yet reconstructing hand-object interaction from a single view and transferring it to robot hands remain difficult, limiting their direct use for robot execution. Prior methods either require task-specific RL training, limiting scalability, or assume clean motion-capture trajectories and thus cannot operate directly on video. We present OmniHOI, a pipeline that turns an RGB video of hand-object interaction into an interaction-faithful trajectory on dexterous hands. The key idea is to enforce physical consistency using the evidence available at each stage: image evidence during reconstruction, contact geometry during retargeting, and dynamics during physics-in-the-loop refinement. Each stage optimizes the corresponding representation directly, correcting errors before they propagate downstream or must be absorbed by a learned policy. Across 150 motion-capture trajectories transferred to each of five dexterous hands with 6 to 22 DoF, we achieve 39-89% success, compared with at most 31% for prior transfer methods. On 60 monocular video clips, we achieve 53% success, compared with 28% for the best prior video-to-robot pipeline. Its trajectories also execute on a real bimanual robot across diverse tasks.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.RO, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2610.10855 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:55:21.992Z dalla pagina https://arxiv.org/list/cs.RO/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).