Metraipolis › archive › dc_63cd832d1ff3

RoboRender: Robot-Oriented Video Generation for Visual Sim-to-Real Transfer — arXiv:2610.09254

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.09254
License (as archived)
CC0-1.0
Archived
2026-10-08T12:48:56.348Z
SHA-256 of the archived text
8c532255702360653266a739c5a344cdb95768d9eb4707564e5fc8162c1e01e4
Stable URL of this page
https://metraipolis.com/documento/dc_63cd832d1ff3

How to cite

RoboRender: Robot-Oriented Video Generation for Visual Sim-to-Real Transfer — arXiv:2610.09254 — arXiv (en), https://arxiv.org/abs/2610.09254 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_63cd832d1ff3 (Metraipolis archive dc_63cd832d1ff3, sha256 8c532255702360653266a739c5a344cdb95768d9eb4707564e5fc8162c1e01e4)
BibTeX
@misc{dc63cd832d1ff3,
  title = {RoboRender: Robot-Oriented Video Generation for Visual Sim-to-Real Transfer — arXiv:2610.09254},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.09254},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_63cd832d1ff3; sha256 8c532255702360653266a739c5a344cdb95768d9eb4707564e5fc8162c1e01e4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.09254. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Simulation enables large-scale, low-cost robot data generation, but policies trained in simulation often fail to transfer to the real world due to the sim-to-real visual discrepancies. Existing approaches often rely on intermediate representations, which can discard rich semantic information or require additional perception modules at deployment. We address this visual sim-to-real gap with RoboRender, a framework that converts simulated trajectories into photorealistic RGB videos for policy learning. RoboRender trains a robot-oriented video generation model conditioned on simulated depth videos, language instructions, and robot RGB mask videos, preserving simulator geometry, robot motion, and action labels while synthesizing realistic textures, backgrounds, and distractors. The generated RGB videos are paired with simulator-provided states and actions to train policies for zero-shot real-world deployment. On robot video test sets, our video model outperforms depth-conditioned video generation baselines in generation quality. In real-world experiments across pick-and-place, articulated-object manipulation, and mobile manipulation tasks, policies trained on RoboRender-generated data achieve a 71% average success rate, outperforming raw simulation renderings and conventional visual domain randomization by approximately 7.1x and 3.6x, respectively. We further show that policy performance improves with more generated videos per simulation trajectory, increasing opening-task success by 65 percentage points. These results demonstrate that generative video rendering mitigates the visual sim-to-real gap for zero-shot policy transfer. Project website: this https URL .

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.RO, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2610.09254 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T12:18:30.509Z dalla pagina https://arxiv.org/list/cs.RO/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).