Metraipolis › archive › dc_f4490ed65710
Q-OIL: Optimistic Exploration in Learning from Human Interventions — arXiv:2611.08576
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.08576
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-09T22:34:25.816Z
- SHA-256 of the archived text
ac7aff953bf9f235fc3739b79f11463cf597085d54a2bc18d43844ffc0018bf5- Stable URL of this page
https://metraipolis.com/documento/dc_f4490ed65710
How to cite
Q-OIL: Optimistic Exploration in Learning from Human Interventions — arXiv:2611.08576 — arXiv (en), https://arxiv.org/abs/2611.08576 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f4490ed65710 (Metraipolis archive dc_f4490ed65710, sha256 ac7aff953bf9f235fc3739b79f11463cf597085d54a2bc18d43844ffc0018bf5)
BibTeX
@misc{dcf4490ed65710,
title = {Q-OIL: Optimistic Exploration in Learning from Human Interventions — arXiv:2611.08576},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.08576},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f4490ed65710; sha256 ac7aff953bf9f235fc3739b79f11463cf597085d54a2bc18d43844ffc0018bf5},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08576. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
As robots are deployed in the real world, they will inevitably make mistakes. Human interventions provide valuable feedback for improving their behavior, enabling them to adapt online to new scenarios. However, studying human-in-the-loop robot learning is difficult: real-world experiments are slow and expensive, so most algorithm development relies on simulated human models. We show that common human modeling assumptions do not match how real people intervene in practice. Through analyzing intervention data from human participants, we find that humans tend to intervene when agent progress stalls, and that their corrective actions are often suboptimal. Motivated by these findings, we introduce a progress-based model of human intervention that achieves higher rank correlation with real-user algorithm rankings, enabling realistic algorithm benchmarking and development in simulation. Building on these insights, we propose Q-OIL, a method that adds optimistic bonuses to intervention transitions in reinforcement learning and softly guides exploration toward human corrections. We find that decoupled critics are critical for stability because they prevent the bonuses from backing up to pre-intervention states. As a result, Q-OIL learns more efficiently than competitive baselines, achieving 10-50% higher success in simulation and on real robots, and improving pretrained VLAs on two bimanual manipulation tasks. As we ground both evaluation and algorithms in realistic human behavior, this work provides a practical path toward scalable robot learning from interventions. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.08576 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-09T22:30:47.300Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).