Metraipolis › archive › dc_b9e2b797c005

Finite-Sample Theory for Fitted Q-Iteration When Actions Are Functions — arXiv:2609.36390

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.36390
License (as archived)
CC0-1.0
Archived
2026-10-07T17:32:34.576Z
SHA-256 of the archived text
b19e086d817b5c8b2fa3d36b1aa9d9adcd574becba3e705c2be47c005aa678dc
Stable URL of this page
https://metraipolis.com/documento/dc_b9e2b797c005

How to cite

Finite-Sample Theory for Fitted Q-Iteration When Actions Are Functions — arXiv:2609.36390 — arXiv (en), https://arxiv.org/abs/2609.36390 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b9e2b797c005 (Metraipolis archive dc_b9e2b797c005, sha256 b19e086d817b5c8b2fa3d36b1aa9d9adcd574becba3e705c2be47c005aa678dc)
BibTeX
@misc{dcb9e2b797c005,
  title = {Finite-Sample Theory for Fitted Q-Iteration When Actions Are Functions — arXiv:2609.36390},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.36390},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b9e2b797c005; sha256 b19e086d817b5c8b2fa3d36b1aa9d9adcd574becba3e705c2be47c005aa678dc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36390. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Offline reinforcement learning seeks optimal decision rules from previously collected data. In some applications, a decision can be an entire function, such as a fluence map in radiation therapy or a smooth movement trajectory in robotics. In this paper, we study the finite-sample theory for fitted Q-iteration (FQI) with functional actions in a discounted infinite-horizon setting. Three major difficulties arise in this setting: first, the absence of a Lebesgue probability density for functional actions complicates coverage descriptions; second, conventional coverage requirements can be restrictive; and third, the large functional action space makes greedy optimization in FQI challenging. To address these difficulties, we study smoothness-regularized policy search under a critic-relative coverage condition. This condition measures how well logged data distinguish relevant action-value differences without requiring an action density. Our main theorem gives finite-sample guarantees for learned-policy regret relative to the best value within a fixed smooth class of functional-action policies. The results allow trajectory lengths to be either bounded or growing and Q-functions to be fitted by either functional-input kernel ridge regression or adaptive functional neural networks. For a few examples, we can obtain polynomially decaying regret bounds in the number of logged transitions, up to logarithmic factors, with logarithmically many FQI iterations. Numerical experiments show gains of learned functional-action policies over constant-action policies and support our adoption of a critic-relative coverage condition.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.LG).
Categorie arXiv: Methodology (stat.ME) ; Machine Learning (cs.LG); Statistics Theory (math.ST); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.36390 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T16:59:48.486Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).