Metraipolis › archive › dc_ea0fcf74f72e
Offline Policy Evaluation via Mixed Bellman Residuals and Adaptive Critic Representations — arXiv:2609.33186
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.33186
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T22:39:57.206Z
- SHA-256 of the archived text
12e296d8e55342696bda663a31e14c8c1a876bb95fc020fd8203b805a319e51c- Stable URL of this page
https://metraipolis.com/documento/dc_ea0fcf74f72e
How to cite
Offline Policy Evaluation via Mixed Bellman Residuals and Adaptive Critic Representations — arXiv:2609.33186 — arXiv (en), https://arxiv.org/abs/2609.33186 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ea0fcf74f72e (Metraipolis archive dc_ea0fcf74f72e, sha256 12e296d8e55342696bda663a31e14c8c1a876bb95fc020fd8203b805a319e51c)
BibTeX
@misc{dcea0fcf74f72e,
title = {Offline Policy Evaluation via Mixed Bellman Residuals and Adaptive Critic Representations — arXiv:2609.33186},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.33186},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ea0fcf74f72e; sha256 12e296d8e55342696bda663a31e14c8c1a876bb95fc020fd8203b805a319e51c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33186. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Evaluating a target policy using data generated by a different behavior policy remains a fundamental challenge in reinforcement learning. While most existing work relies on the standard one-step Bellman residual, we consider a convex combination of one-step and two-step residuals with a fixed mixing weight. In ideal settings, this mixed Bellman formulation can provide a natural bias--variance trade-off between approximation error under a restricted value-function class and the increased variance arising from multi-step importance weighting. To solve this mixed residual optimization, we adopt a minimax formulation involving a critic function. Unlike standard approaches that rely on a fixed functional class, we construct a data-dependent critic representation using predicted future feature directions which effectively induces a kernel adapted to the underlying transition dynamics. This allows the critic to focus on directions that are most relevant for the estimated Bellman error. To control overfitting, we use sample splitting to construct the critic and estimate the value function on separate data subsets. Simulation studies and MetaWorld tasks illustrate the effect of the mixing parameter and show that intermediate residual combinations can improve value estimation in challenging settings. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG). Categorie arXiv: Machine Learning (stat.ML) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.33186 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T22:14:28.696Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).