Metraipolis › archive › dc_7425baeac148

Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-Tuning — arXiv:2608.10473

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.10473
License (as archived)
CC0-1.0
Archived
2026-10-19T06:09:11.639Z
SHA-256 of the archived text
b825f534e784e3732474a51e6c42b2038c93b9b9ff3750bd725e1547d87c50ff
Stable URL of this page
https://metraipolis.com/documento/dc_7425baeac148

How to cite

Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-Tuning — arXiv:2608.10473 — arXiv (en), https://arxiv.org/abs/2608.10473 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7425baeac148 (Metraipolis archive dc_7425baeac148, sha256 b825f534e784e3732474a51e6c42b2038c93b9b9ff3750bd725e1547d87c50ff)
BibTeX
@misc{dc7425baeac148,
  title = {Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-Tuning — arXiv:2608.10473},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.10473},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7425baeac148; sha256 b825f534e784e3732474a51e6c42b2038c93b9b9ff3750bd725e1547d87c50ff},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.10473. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Offline-to-online (O2O) reinforcement learning aims to leverage policies pretrained on static datasets while improving them through online interaction. However, directly reusing an offline-trained critic can hinder online fine-tuning: as the policy and data distribution change rapidly, value estimates inherited from offline training may become misaligned with the online environment, leading to inaccurate policy improvement and inefficient exploration. To address this problem, we introduce Critic-Free Pretraining: an efficient paradigm that completely abandons the approach of offline critic training, allowing a freshly initialized critic to adapt without inheriting biased estimates. CFP is compatible with various mainstream O2O algorithms and consistently matches or improves upon conventional O2O algorithms across a diverse set of tasks, with particularly pronounced gains on several challenging tasks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-12 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.10473 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:57:28.370Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).