Metraipolis › archive › dc_974a8fc62034
Echo: Scaling Visual Reinforcement Learning with Off-policy Experience — arXiv:2610.37865
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.37865
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T06:11:07.185Z
- SHA-256 of the archived text
5c59267b355026b15aa3b77087cc89a71922d3dbca0437cde54b3b6455eeb8f6- Stable URL of this page
https://metraipolis.com/documento/dc_974a8fc62034
How to cite
Echo: Scaling Visual Reinforcement Learning with Off-policy Experience — arXiv:2610.37865 — arXiv (en), https://arxiv.org/abs/2610.37865 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_974a8fc62034 (Metraipolis archive dc_974a8fc62034, sha256 5c59267b355026b15aa3b77087cc89a71922d3dbca0437cde54b3b6455eeb8f6)
BibTeX
@misc{dc974a8fc62034,
title = {Echo: Scaling Visual Reinforcement Learning with Off-policy Experience — arXiv:2610.37865},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.37865},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_974a8fc62034; sha256 5c59267b355026b15aa3b77087cc89a71922d3dbca0437cde54b3b6455eeb8f6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.37865. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Scaling visual reinforcement learning (RL) with GPU-accelerated simulation is computationally expensive due to rendering and visual-policy inference costs. Consequently, two-stage teacher--student distillation is commonly used to train visual policies, but introduces an imitation gap. In this preliminary study, we introduce \echomethod{}, \textbf{a single-stage off-policy RL framework that scales experience collection without proportionally increasing visual computation}. Through \textbf{inverse distillation}, a state-based auxiliary policy imitates the evolving visual policy and collects inexpensive, unrendered experience. Echo leverages this experience for off-policy value learning, directly improving the visual policy through a shared privileged critic. We instantiate \echomethod{} with FastTD3 and evaluate it on three visual control tasks and two dexterous manipulation tasks in Genesis. Relative to \sdpgmethod{} under the evaluated configurations, Echo reduces time to reference returns by an average of \textbf{60\% across the three locomotion tasks} and \textbf{56\% across both hand tasks}, while improving peak mean returns by \textbf{62\% and 12\%}, respectively. Both hand tasks also achieve higher mean returns at matched simulator-transition budgets. These initial results suggest that scaling off-policy experience is a promising approach to efficient direct visual RL.
Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.37865 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T06:07:08.601Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).