Metraipolis › archive › dc_cc5b374bc8e6

From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention — arXiv:2609.21788

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.21788
License (as archived)
CC0-1.0
Archived
2026-10-10T01:47:36.752Z
SHA-256 of the archived text
614ff9b0d9fa1df5f7f2558555e8f809dedfc3756408051aa27268aa1b2a6f4c
Stable URL of this page
https://metraipolis.com/documento/dc_cc5b374bc8e6

How to cite

From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention — arXiv:2609.21788 — arXiv (en), https://arxiv.org/abs/2609.21788 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cc5b374bc8e6 (Metraipolis archive dc_cc5b374bc8e6, sha256 614ff9b0d9fa1df5f7f2558555e8f809dedfc3756408051aa27268aa1b2a6f4c)
BibTeX
@misc{dccc5b374bc8e6,
  title = {From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention — arXiv:2609.21788},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.21788},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cc5b374bc8e6; sha256 614ff9b0d9fa1df5f7f2558555e8f809dedfc3756408051aa27268aa1b2a6f4c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.21788. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A pretrained robot foundation policy may execute most of a long-horizon task yet repeatedly fail at a few critical subtasks. Collecting additional full-task demonstrations for supervised fine-tuning (SFT) requires operators to repeat behaviors the policy already performs well. Reinforcement learning (RL) fine-tuning offers a promising path to bridge this gap, but existing approaches struggle to solve long-horizon tasks using only sparse rewards. We present PARTS (Policy Adaptation with RL on Targeted Subtasks), a real-world subtask RL framework that concentrates practice at these bottlenecks while allowing training rollouts to proceed with minimal human intervention. The frozen pretrained policy supplies nominal actions throughout execution, while agent-generated selectors and success verifiers activate residual corrections and provide local outcome rewards. These rewards support learning from successful subtasks even when complete-task successes are scarce. Training combines online RL with success-reweighted retraining, and each retrained residual policy is redeployed to collect further experience. Humans identify bottlenecks during setup and perform physical resets when needed. On bimanual YAM and single-arm Franka tasks, PARTS improves complete-task success from 32% to 61% and from 50% to 95%, respectively, using tens of minutes of real-world RL rollouts per task on average. Compared with existing real-world RL fine-tuning methods, PARTS raises full-task success by more than 25% under the same robot-rollout budget while requiring less human involvement.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-21 in cs.LG).
Categorie arXiv: Robotics (cs.RO) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.21788 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T01:14:16.717Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).