Metraipolis › archive › dc_ce41088bbb2d
Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent — arXiv:2610.35817
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.35817
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T08:03:28.228Z
- SHA-256 of the archived text
defb231617a6971e89b8afcc047538e7ab06285c83a110e3b96c8d2dc54c19b5- Stable URL of this page
https://metraipolis.com/documento/dc_ce41088bbb2d
How to cite
Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent — arXiv:2610.35817 — arXiv (en), https://arxiv.org/abs/2610.35817 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ce41088bbb2d (Metraipolis archive dc_ce41088bbb2d, sha256 defb231617a6971e89b8afcc047538e7ab06285c83a110e3b96c8d2dc54c19b5)
BibTeX
@misc{dcce41088bbb2d,
title = {Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent — arXiv:2610.35817},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.35817},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ce41088bbb2d; sha256 defb231617a6971e89b8afcc047538e7ab06285c83a110e3b96c8d2dc54c19b5},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.35817. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
A robot policy needs to understand what a task is asking for in a scene, and how to move the robot to accomplish that task. To achieve these two skills, we introduce Zero-real Agent Policy (ZAP), a recipe that teaches a small off-the-shelf vision-language model to control a robot via a simple tool harness. We use a pretrained 27B parameter model to bootstrap world understanding and reasoning, followed by supervised fine-tuning and GRPO on simple simulated tasks to teach it how to move the robot. Without any real-world robot data during training, ZAP reaches 96.7% success on real-world tasks, matching GPT-6 Astra in our harness and outperforming the MolmoAct2 VLA (33.3%), while running 3x faster and 17x cheaper than GPT-6 Astra. We demonstrate that ZAP also generalizes to completely new objects, goals, strategies, and language by seating a test tube in a rack (and choosing a different hole in the rack when prompted), handing a test tube between arms mid-air, drawing an X on a tic-tac-toe board, and playing chess. These goals and types of manipulation don't appear in our training data. Furthermore, ZAP also transfers to an unseen embodiment. On a single-arm Franka, ZAP reaches 55.9% on RoboLab-120, 4.9 percentage points ahead of the leading VLA/WAM baseline. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.35817 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:55:31.018Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).