Metraipolis › archive › dc_b2b4deadaf11

Real2Gym: Building Gyms from Videos, Bringing Skills to Robots — arXiv:2609.37089

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.37089
License (as archived)
CC0-1.0
Archived
2026-10-07T18:40:54.996Z
SHA-256 of the archived text
53f0682705f1911ef63850ea8351f9edf944d86a79429b5bf423274e88108f5f
Stable URL of this page
https://metraipolis.com/documento/dc_b2b4deadaf11

How to cite

Real2Gym: Building Gyms from Videos, Bringing Skills to Robots — arXiv:2609.37089 — arXiv (en), https://arxiv.org/abs/2609.37089 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b2b4deadaf11 (Metraipolis archive dc_b2b4deadaf11, sha256 53f0682705f1911ef63850ea8351f9edf944d86a79429b5bf423274e88108f5f)
BibTeX
@misc{dcb2b4deadaf11,
  title = {Real2Gym: Building Gyms from Videos, Bringing Skills to Robots — arXiv:2609.37089},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37089},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b2b4deadaf11; sha256 53f0682705f1911ef63850ea8351f9edf944d86a79429b5bf423274e88108f5f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37089. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Real-world videos provide rich demonstrations of manipulation, but turning them into reusable robot skills requires visually aligned environments, executable physical interactions, and mechanisms for learning from experience. We introduce Real2Gym, an agentic Real2Sim2Real framework that turns human and robot demonstrations into interactive simulation gyms and brings skills acquired in simulation to physical robots. The Real2Sim module reconstructs editable scenes, aligns objects and cameras with the input, validates demonstrated or retargeted actions through native physics execution, and generates task-conditioned variations with action-feasibility checks. Within these environments, the agent generates executable code for manipulation stages, observes their outcomes, and distills successful attempts and failures into reusable task procedures, object-relative motions, and recovery strategies. Through a shared perception-and-control interface, these skills guide subsequent execution in simulation and on real robots, with motions adapted to current observations and no updates to the underlying model weights. Extensive evaluations demonstrate that Real2Gym enables high-fidelity simulation environment reconstruction, outperforming GPT-6 Astra Direct Mode by 16.7% in success rate with approximately 74.9% fewer policy-execution tokens across these environments, while exceeding it by 33.3% in physical robot execution success rate across four tasks on a real Franka robot.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-30 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.37089 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T18:04:24.817Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).