Metraipolis › archive › dc_719bde81fb2a

Efficient Real-World Online Reinforcement Learning for Robot Manipulation via Centralized Training and Critic Decomposition — arXiv:2608.09762

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.09762
License (as archived)
CC0-1.0
Archived
2026-10-19T09:12:53.829Z
SHA-256 of the archived text
a791f9bf21f64766709fcd258a919ad9a59b00afa986cda304ea09a50433194c
Stable URL of this page
https://metraipolis.com/documento/dc_719bde81fb2a

How to cite

Efficient Real-World Online Reinforcement Learning for Robot Manipulation via Centralized Training and Critic Decomposition — arXiv:2608.09762 — arXiv (en), https://arxiv.org/abs/2608.09762 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_719bde81fb2a (Metraipolis archive dc_719bde81fb2a, sha256 a791f9bf21f64766709fcd258a919ad9a59b00afa986cda304ea09a50433194c)
BibTeX
@misc{dc719bde81fb2a,
  title = {Efficient Real-World Online Reinforcement Learning for Robot Manipulation via Centralized Training and Critic Decomposition — arXiv:2608.09762},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.09762},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_719bde81fb2a; sha256 a791f9bf21f64766709fcd258a919ad9a59b00afa986cda304ea09a50433194c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.09762. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Real-world online reinforcement learning (RL) provides a promising approach for training robotic manipulation policies directly in the physical world, avoiding the sim-to-real gap and enabling continuous policy refinement through human-in-the-loop interaction. Recent methods have demonstrated sample-efficient learning through human intervention but remain limited to small randomization ranges and encounter challenges with the non-stationarity induced by concurrently training multiple agents. To address these limitations, we introduce a unified framework that combines centralized training with decentralized execution (CTDE) and a Hybrid Reward Architecture (HRA). This enables multiple actors to share a centralized multi-head critic. The critic is decomposed into task and grasp heads, corresponding to the sparse task reward and a potential-based grasping reward, respectively. We accordingly reformulate the critic and actor objectives to exploit the decomposed Q-values while explicitly accounting for the categorical action distribution of the discrete gripper policy. Experimental results demonstrate that the proposed framework substantially improves both sample efficiency and policy performance. We validate our approach on two robotic arms and a simulated humanoid robot across tennis ball and banana pick-and-place, pot reset, and simulated block relocation tasks under dimension-wise domain randomization, approximately 5-25x larger than those considered in prior work. Compared with a state-of-the-art baseline, our method improves the success rate from 60% to 80% on tennis ball pick-and-place, from 60% to 90% on banana pick-and-place, and from 25% to 95% on simulated block relocation, while also successfully accomplishing a task where the baseline consistently fails. Videos and more details are available at our project website: this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-11 in cs.RO).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2608.09762 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T09:11:58.797Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).