Metraipolis › archive › dc_78c021c97bf6

Distributionally Robust Average-Reward Reinforcement Learning: Finite-Sample Guarantees under Weak Communication — arXiv:2609.32727

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.32727
License (as archived): CC0-1.0
Archived: 2026-10-07T22:36:47.590Z
SHA-256 of the archived text: 6474ebe833e7bffcd3bdba186b1dae6e75afaf2c38b495258e13dd25399a3fd1
Stable URL of this page: https://metraipolis.com/documento/dc_78c021c97bf6

How to cite

Distributionally Robust Average-Reward Reinforcement Learning: Finite-Sample Guarantees under Weak Communication — arXiv:2609.32727 — arXiv (en), https://arxiv.org/abs/2609.32727 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_78c021c97bf6 (Metraipolis archive dc_78c021c97bf6, sha256 6474ebe833e7bffcd3bdba186b1dae6e75afaf2c38b495258e13dd25399a3fd1)

BibTeX

@misc{dc78c021c97bf6,
  title = {Distributionally Robust Average-Reward Reinforcement Learning: Finite-Sample Guarantees under Weak Communication — arXiv:2609.32727},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32727},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_78c021c97bf6; sha256 6474ebe833e7bffcd3bdba186b1dae6e75afaf2c38b495258e13dd25399a3fd1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32727. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We study distributionally robust reinforcement learning (DR-RL) in the average-reward setting under weak communication. Our main result provides finite-sample guarantees for estimating the robust optimal average reward and learning a near-optimal policy, covering both SA-rectangular and S-rectangular structures with divergence-based and distance-based uncertainty sets. Specifically, for Kullback--Leibler and $f_k$-divergence balls, we establish explicit radius conditions under which the robust average-reward Bellman equation admits a constant-gain solution, while for total variation and Wasserstein balls, any positive radius suffices without requiring the nominal MDP to be weakly communicating. Our algorithm is prior-knowledge-free and achieves sample complexities of $\widetilde O(|\mathcal{S}||\mathcal{A}|p_{\wedge}^{-1}\operatorname{Span}^{2}(u_{\delta}^{\ast})\epsilon^{-2})$ for estimating the robust optimal average reward and $\widetilde O(|\mathcal{S}||\mathcal{A}|p_{\wedge}^{-2}\operatorname{Span}^{2}(u_{\delta}^{\ast})\epsilon^{-2})$ for learning an $\epsilon$-optimal policy. Here, $p_{\wedge}$ is the smallest positive nominal transition probability and $u_{\delta}^{\ast}$ is a robust optimal bias function. We further provide an almost-tight explicit upper bound on $\operatorname{Span}(u_{\delta}^{\ast})$. Finally, we validate the predicted $n^{-1/2}$ convergence rate through numerical experiments.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Optimization and Control (math.OC); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.32727 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.749Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).