Metraipolis › archive › dc_d0a94cdc7b60
Q-Learning with Stable Infinite-Dimensional Linear Function Approximation — arXiv:2608.22636
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.22636
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T08:25:47.157Z
- SHA-256 of the archived text
d35cfc6bffdcd9c00d5c3a84b4abfac72740706c0dadf4b0dc08ddef55506036- Stable URL of this page
https://metraipolis.com/documento/dc_d0a94cdc7b60
How to cite
Q-Learning with Stable Infinite-Dimensional Linear Function Approximation — arXiv:2608.22636 — arXiv (en), https://arxiv.org/abs/2608.22636 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d0a94cdc7b60 (Metraipolis archive dc_d0a94cdc7b60, sha256 d35cfc6bffdcd9c00d5c3a84b4abfac72740706c0dadf4b0dc08ddef55506036)
BibTeX
@misc{dcd0a94cdc7b60,
title = {Q-Learning with Stable Infinite-Dimensional Linear Function Approximation — arXiv:2608.22636},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.22636},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d0a94cdc7b60; sha256 d35cfc6bffdcd9c00d5c3a84b4abfac72740706c0dadf4b0dc08ddef55506036},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.22636. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Q-learning with linear function approximation can be unstable because an arbitrary approximation architecture need not preserve the Bellman contraction. We develop a stable infinite-dimensional linear function approximation framework for Q-learning from a single Markovian behavior-policy trajectory. The learning variable is a coefficient field $\theta\in C(\mathbb L)$ on a compact latent metric space $(\mathbb L,\rho)$. The framework uses a reconstruction operator that maps $\theta$ to a continuous Q-function and a compression operator that maps Bellman updates back to latent coordinates. Nonexpansiveness of both operators induces a contractive latent Bellman map on $C(\mathbb L)$, with a unique fixed point $\theta^*$ whose reconstruction approximates the optimal Q-function up to representation error. We propose two stochastic approximation (SA) algorithms and establish their sup-norm convergence bounds with a leading term of order $\widetilde O(n^{-1/2})$. The infinite-dimensional formulation provides a powerful abstraction for identifying the structures that govern statistical difficulty. Smoothness of the compression map in $\rho$ is inherited by $\theta^*$ and the SA iterates, allowing uniform estimation errors to be controlled through covering numbers of $(\mathbb L,\rho)$ rather than the dimension of $C(\mathbb L)$. Remarkably, the SA algorithms we propose are agnostic to the choice of $\rho$, and thus can automatically adapt to both the smoothness and the geometry. We further illustrate the framework through Q-measure-learning with linear density approximation and output-layer neural weight training under a frozen pretrained network.
Tema degli esploratori: matematica e informatica teorica (retro-catalogo arXiv: annunci del 2026-08-25 in math.OC).
Categorie arXiv: Machine Learning (cs.LG) ; Optimization and Control (math.OC); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2608.22636 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T08:18:05.718Z dalla pagina https://arxiv.org/catchup/math.OC/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).