Metraipolis › archive › dc_4b30daaf219a
VidAct: Learning Manipulation from In-the-Wild Videos with Object-Centric 3D Awareness — arXiv:2609.36870
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36870
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T19:15:24.511Z
- SHA-256 of the archived text
0d8dadccac18e9f86d0cb8cc4c4a357f0e84d510d30663ba5cfc531b21f575d3- Stable URL of this page
https://metraipolis.com/documento/dc_4b30daaf219a
How to cite
VidAct: Learning Manipulation from In-the-Wild Videos with Object-Centric 3D Awareness — arXiv:2609.36870 — arXiv (en), https://arxiv.org/abs/2609.36870 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4b30daaf219a (Metraipolis archive dc_4b30daaf219a, sha256 0d8dadccac18e9f86d0cb8cc4c4a357f0e84d510d30663ba5cfc531b21f575d3)
BibTeX
@misc{dc4b30daaf219a,
title = {VidAct: Learning Manipulation from In-the-Wild Videos with Object-Centric 3D Awareness — arXiv:2609.36870},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36870},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4b30daaf219a; sha256 0d8dadccac18e9f86d0cb8cc4c4a357f0e84d510d30663ba5cfc531b21f575d3},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36870. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Video demonstrations offer a scalable alternative to costly robot data for learning manipulation, yet existing reconstruction-based approaches often rely on constrained camera viewpoints or human-to-robot retargeting, while the reconstructed trajectories are difficult to adapt to new objects configurations without distorting the trajectory shape. Another key limitation is that the resulting policies often lack precise object-level 3D geometry awareness, limiting object grounding and object shape awareness critical for precise manipulation. To bridge these gaps, we propose VidAct, an efficient video-to-robot framework that learns object-centric, 3D-aware manipulation policies from a single monocular video per task and enables zero-shot real-world deployment. VidAct consists of three key components. First, VidAct reconstructs object meshes and motion from arbitrary demo videos and canonicalizes the motion in the static object frame, avoiding embodiment-specific retargeting and accommodating diverse camera viewpoints. Second, VidAct employ residual trajectory transfer for adapting the reconstructed motion to novel object configurations while preserving its motion shape. Finally, as the key policy-learning component, VidAct predicts simulation-provided privileged complete-object point clouds at each frame as an auxiliary task while retaining RGB-only deployment, providing dense object-centric supervision over both object pose and 3D geometry. Experiments on human, robot, generated, and internet videos demonstrate broad video applicability and zero-shot deployment. Per-frame complete-object 3D supervision improves policy generalization and sim-to-real success, while residual trajectory transfer enables reliable trajectory adaptation with better shape preservation. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-30 in cs.RO). Categorie arXiv: Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2609.36870 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T18:40:59.458Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).