Metraipolis › archive › dc_e613c3ec7d69
Adaptive Expert Guidance for Efficient On-Policy Reinforcement Learning — arXiv:2610.06019
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.06019
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:27:21.535Z
- SHA-256 of the archived text
9c9956af4e7eac5b8268fe925a5088f64cd96238c2f9f9ed2674114fe55b8a77- Stable URL of this page
https://metraipolis.com/documento/dc_e613c3ec7d69
How to cite
Adaptive Expert Guidance for Efficient On-Policy Reinforcement Learning — arXiv:2610.06019 — arXiv, https://arxiv.org/abs/2610.06019 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e613c3ec7d69 (Metraipolis archive dc_e613c3ec7d69, sha256 9c9956af4e7eac5b8268fe925a5088f64cd96238c2f9f9ed2674114fe55b8a77)
BibTeX
@misc{dce613c3ec7d69,
title = {Adaptive Expert Guidance for Efficient On-Policy Reinforcement Learning — arXiv:2610.06019},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.06019},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e613c3ec7d69; sha256 9c9956af4e7eac5b8268fe925a5088f64cd96238c2f9f9ed2674114fe55b8a77},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.06019. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
With massively parallel simulation, on-policy Reinforcement Learning methods such as PPO have become standard in many domains. However, learning from scratch is sample-inefficient and fails to exploit the potential existence of a suboptimal expert, such as a heuristic, a model-based controller, or a policy trained on a related task. Such an expert is often available and can guide early training, but its sub-optimality limits final performance. The challenge then becomes balancing expert guidance against learning from rewards. Existing methods set the expert's influence through a blending weight, a schedule, or an evaluation-driven curriculum. Alternatively, they adapt it with additional learned components such as critics over expert actions or auxiliary agents. However, none optimizes it using the same on-policy objective as the policy itself. We propose a method in which the learner and the expert alternate control within each training episode, and the expert's share of control is a single learnable parameter optimized jointly with the policy. The learner benefits from the expert early in training, but its share of control declines as the learner becomes more competent, until eventually vanishing completely. This leaves the learner acting alone and better than the suboptimal expert. We evaluate our method on 34 tasks across two benchmarks, spanning discrete and continuous action spaces, using both learned and model-based experts. Our method improves sample efficiency over guided and unguided baselines while requiring minimal hyperparameter variation. The expert's share decays to zero as the learner improves, vanishing when the expert is no longer useful. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.06019 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:06.103Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).