Metraipolis › archive › dc_2001a862542f
Dynamics Models for Offline Hyperparameter Selection in Real-World RL — arXiv:2608.11349
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.11349
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T05:15:49.402Z
- SHA-256 of the archived text
6edfdcb788935f655ee33d223feef5c8253ea1bbd3b432eaf27dae9a9c9028fc- Stable URL of this page
https://metraipolis.com/documento/dc_2001a862542f
How to cite
Dynamics Models for Offline Hyperparameter Selection in Real-World RL — arXiv:2608.11349 — arXiv (en), https://arxiv.org/abs/2608.11349 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2001a862542f (Metraipolis archive dc_2001a862542f, sha256 6edfdcb788935f655ee33d223feef5c8253ea1bbd3b432eaf27dae9a9c9028fc)
BibTeX
@misc{dc2001a862542f,
title = {Dynamics Models for Offline Hyperparameter Selection in Real-World RL — arXiv:2608.11349},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.11349},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2001a862542f; sha256 6edfdcb788935f655ee33d223feef5c8253ea1bbd3b432eaf27dae9a9c9028fc},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11349. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
A key obstacle to deploying reinforcement learning in real-world systems is hyperparameter selection, particularly when simulators are unavailable and online experimentation is costly. Prior work has proposed calibration models trained on offline data to approximate environment dynamics and enable offline hyperparameter selection, but these methods have so far been evaluated only in simple simulated settings. In this paper, we present the first application of calibration models in a real-world industrial setting: a municipal water treatment plant. We evaluate several calibration model approaches, including a k-nearest neighbors model with a Laplacian distance metric, on high-dimensional, non-stationary sensor data for nexting prediction tasks. Our results show that these models can generate realistic long-horizon rollouts and recover meaningful hyperparameter sensitivity trends. We further examine how calibration models scale to year-long datasets, how they support the selection of fine-tuning learning rates for pre-trained agents, and how robust they are under distribution shift. Overall, our findings provide a proof of concept for using offline dynamics models to support RL deployment in real-world environments, while highlighting important practical challenges for future work. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.11349 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T05:08:46.683Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).