Metraipolis › archive › dc_7425baeac148
Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-Tuning — arXiv:2608.10473
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.10473
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T06:09:11.639Z
- SHA-256 of the archived text
b825f534e784e3732474a51e6c42b2038c93b9b9ff3750bd725e1547d87c50ff- Stable URL of this page
https://metraipolis.com/documento/dc_7425baeac148
How to cite
Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-Tuning — arXiv:2608.10473 — arXiv (en), https://arxiv.org/abs/2608.10473 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7425baeac148 (Metraipolis archive dc_7425baeac148, sha256 b825f534e784e3732474a51e6c42b2038c93b9b9ff3750bd725e1547d87c50ff)
BibTeX
@misc{dc7425baeac148,
title = {Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-Tuning — arXiv:2608.10473},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.10473},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7425baeac148; sha256 b825f534e784e3732474a51e6c42b2038c93b9b9ff3750bd725e1547d87c50ff},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.10473. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Offline-to-online (O2O) reinforcement learning aims to leverage policies pretrained on static datasets while improving them through online interaction. However, directly reusing an offline-trained critic can hinder online fine-tuning: as the policy and data distribution change rapidly, value estimates inherited from offline training may become misaligned with the online environment, leading to inaccurate policy improvement and inefficient exploration. To address this problem, we introduce Critic-Free Pretraining: an efficient paradigm that completely abandons the approach of offline critic training, allowing a freshly initialized critic to adapt without inheriting biased estimates. CFP is compatible with various mainstream O2O algorithms and consistently matches or improves upon conventional O2O algorithms across a diverse set of tasks, with particularly pronounced gains on several challenging tasks. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-12 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.10473 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T05:57:28.370Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).