Metraipolis › archive › dc_7402db0b23f4
Policy Plasticity Matters in Offline-to-Online Reinforcement Learning: Refitting Offline Policies for Online Adaptation — arXiv:2609.33127
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.33127
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T21:01:02.824Z
- SHA-256 of the archived text
afe0f1e580a137f4cb4dca8d430ee78581ca3c3885ba218b05e391868b7cb9da- Stable URL of this page
https://metraipolis.com/documento/dc_7402db0b23f4
How to cite
Policy Plasticity Matters in Offline-to-Online Reinforcement Learning: Refitting Offline Policies for Online Adaptation — arXiv:2609.33127 — arXiv (en), https://arxiv.org/abs/2609.33127 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7402db0b23f4 (Metraipolis archive dc_7402db0b23f4, sha256 afe0f1e580a137f4cb4dca8d430ee78581ca3c3885ba218b05e391868b7cb9da)
BibTeX
@misc{dc7402db0b23f4,
title = {Policy Plasticity Matters in Offline-to-Online Reinforcement Learning: Refitting Offline Policies for Online Adaptation — arXiv:2609.33127},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.33127},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7402db0b23f4; sha256 afe0f1e580a137f4cb4dca8d430ee78581ca3c3885ba218b05e391868b7cb9da},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33127. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Offline-to-Online Reinforcement Learning (O2O RL) has emerged as a practical paradigm that pre-trains the policy using static offline datasets and subsequently adapts the policy through online interactions. Existing O2O methods primarily address the transition through value calibration, while generally treating the offline-trained policy as a given initialization. We instead study O2O adaptation from the perspective of network plasticity, asking whether the offline-trained policy remains sufficiently adaptable for online learning. Controlled experiments show that prolonged optimization on static offline data progressively reduces network plasticity even after offline performance has largely saturated, and that lower plasticity is associated with weaker subsequent online improvement. Motivated by these observations, we propose REstoring plasticity via Fresh Initialization and policy Transfer (REFIT), a lightweight model-level method for the O2O transition. Before online fine-tuning, REFIT distills the offline policy into a freshly initialized student while temporarily freezing a random subset of student units, transferring the learned offline behavior to a more plastic policy initialization. Extensive experiments on D4RL and OGBench demonstrate that REFIT consistently achieves higher aggregate performance than existing O2O plug-in methods across both Cal-QL and IQL backbones, while plasticity diagnostics and ablations provide further evidence of restored network plasticity. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.33127 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:02.553Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).