Metraipolis › archive › dc_83edf404ba65
Role-Adaptive Policy Optimization for Offline Reinforcement Learning — arXiv:2609.40149
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.40149
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T13:46:57.953Z
- SHA-256 of the archived text
fe4566d3707980a3b2a6a37ff7c1bbb335d203d1f9e1787751725cf7198268de- Stable URL of this page
https://metraipolis.com/documento/dc_83edf404ba65
How to cite
Role-Adaptive Policy Optimization for Offline Reinforcement Learning — arXiv:2609.40149 — arXiv (en), https://arxiv.org/abs/2609.40149 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_83edf404ba65 (Metraipolis archive dc_83edf404ba65, sha256 fe4566d3707980a3b2a6a37ff7c1bbb335d203d1f9e1787751725cf7198268de)
BibTeX
@misc{dc83edf404ba65,
title = {Role-Adaptive Policy Optimization for Offline Reinforcement Learning — arXiv:2609.40149},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.40149},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_83edf404ba65; sha256 fe4566d3707980a3b2a6a37ff7c1bbb335d203d1f9e1787751725cf7198268de},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.40149. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Policy regularization in offline reinforcement learning balances policy improvement against reliance on uncertain value estimates. This balance can differ between selecting actions for execution and supplying actions for critic bootstrapping, yet methods such as TD3+BC couple these roles through a shared policy. We propose Role-Adaptive Policy Optimization (RAPO), which adapts policy-update coefficients according to their roles in value learning and execution. RAPO learns these coefficients by differentiating through candidate policy updates formed using the base algorithm's actor objective. For TD3+BC, RAPO separates bootstrap and execution actors and adapts their coefficients independently: the bootstrap objective penalizes policy-induced changes in target values, while the execution objective evaluates a local policy-improvement surrogate. For IQL, whose value learning is already independent of the execution actor, RAPO preserves the original value updates and adapts only the inverse temperature in advantage-weighted policy extraction. Experiments on D4RL locomotion and AntMaze tasks show improvements over both base algorithms, with larger gains for TD3+BC, whose RAPO instantiation outperforms baselines on average. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.40149 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T13:18:10.518Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).