Metraipolis › archive › dc_e276709eae46

Non-asymptotic Convergence of Average-reward Q-learning with Options — arXiv:2610.20347

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.20347
License (as archived)
CC0-1.0
Archived
2026-10-15T14:20:10.263Z
SHA-256 of the archived text
ef23cfa06e5837bf4a6e03c92b1d677fed56c3eb9458032bb2037e98ffa96c28
Stable URL of this page
https://metraipolis.com/documento/dc_e276709eae46

How to cite

Non-asymptotic Convergence of Average-reward Q-learning with Options — arXiv:2610.20347 — arXiv (en), https://arxiv.org/abs/2610.20347 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e276709eae46 (Metraipolis archive dc_e276709eae46, sha256 ef23cfa06e5837bf4a6e03c92b1d677fed56c3eb9458032bb2037e98ffa96c28)
BibTeX
@misc{dce276709eae46,
  title = {Non-asymptotic Convergence of Average-reward Q-learning with Options — arXiv:2610.20347},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.20347},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e276709eae46; sha256 ef23cfa06e5837bf4a6e03c92b1d677fed56c3eb9458032bb2037e98ffa96c28},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.20347. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We study the finite-sample convergence of Q-learning algorithms for the average-reward formulation of Reinforcement Learning (RL) with options. While options allow temporarily extended and hierarchical actions, the average-reward formulation incorporates continual tasks beyond discounted and episodic RL. We establish the first $\tilde{O}(1/\text{iterations})$ convergence guarantees, in the mean-square sense, for average-reward option Q-learning algorithms in both the inter- and intra-option settings. First, we prove that both inter- and intra-option Bellman operators satisfy multi-step span contractions under unichain assumptions. Then, we propose synchronous estimators of these multi-step option Bellman operators leading to two variants of option Q-learning. Finally, we extend the stochastic approximation framework to prove convergence rates of these two algorithms under the minimal assumptions used in the literature.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.20347 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:53:35.734Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).