Metraipolis › archive › dc_dc1168131ca5
Max-Q Selective Imitation for Human-in-the-Loop Online Robot Learning — arXiv:2608.15088
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.15088
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T17:57:47.126Z
- SHA-256 of the archived text
b0fc28a48121a4682fb68b72b0c937092d60a26dd4083c5087b1e986261182dc- Stable URL of this page
https://metraipolis.com/documento/dc_dc1168131ca5
How to cite
Max-Q Selective Imitation for Human-in-the-Loop Online Robot Learning — arXiv:2608.15088 — arXiv (en), https://arxiv.org/abs/2608.15088 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dc1168131ca5 (Metraipolis archive dc_dc1168131ca5, sha256 b0fc28a48121a4682fb68b72b0c937092d60a26dd4083c5087b1e986261182dc)
BibTeX
@misc{dcdc1168131ca5,
title = {Max-Q Selective Imitation for Human-in-the-Loop Online Robot Learning — arXiv:2608.15088},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.15088},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dc1168131ca5; sha256 b0fc28a48121a4682fb68b72b0c937092d60a26dd4083c5087b1e986261182dc},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15088. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Human-in-the-loop (HIL) online reinforcement learning for real robots must absorb human interventions quickly while continuing to improve beyond the human prior. We present a training method for this setting based on two components. First, an \emph{MC Q-chunk} critic regresses chunk-level action values onto Monte Carlo returns from the replay buffer, performing sample-average (behavior) policy evaluation so that intervention trajectories are credited directly rather than diluted by current-policy TD backups. Second, \emph{max-Q selective imitation} updates the actor by imitating, at each state, the higher-$Q$ action between the current policy action and a buffer sample under a hard winner-take-all rule. This rule automatically switches between learning from interventions and on-policy self-improvement: when the autonomous policy is stronger, targets align with the policy distribution, reducing the policy--target-sample gap that otherwise induces execution-time distribution shift. In practice we score candidates with a standard critic ensemble mean to reduce comparison noise, without softening targets or introducing score-gap thresholds. On a real USB pick-and-insertion task with 20 demonstrations, ACT QChunk-MCBC attains 99\% success within 30 minutes of HIL training, whereas HIL-SERL requires about 5 hours to converge. In simulation on Peg Insertion and Square, ACT/Flow Q-chunk variants similarly reach $\ge$96\% success within roughly half an hour of effective training, outperforming HIL-SERL, EXPO, and E2HiL on the success--time frontier.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.15088 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T16:29:13.983Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).