Metraipolis › archive › dc_d18ce1d47422
A Convergence Framework for Deep $V$-Learning: Error Propagation and Sharp Action-Gap Bounds — arXiv:2609.18782
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.18782
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T15:04:53.341Z
- SHA-256 of the archived text
3a3eb174e826993f82e0004a71e88a788ccc80bcfc203c4e196a25218e5fd80b- Stable URL of this page
https://metraipolis.com/documento/dc_d18ce1d47422
How to cite
A Convergence Framework for Deep $V$-Learning: Error Propagation and Sharp Action-Gap Bounds — arXiv:2609.18782 — arXiv (en), https://arxiv.org/abs/2609.18782 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d18ce1d47422 (Metraipolis archive dc_d18ce1d47422, sha256 3a3eb174e826993f82e0004a71e88a788ccc80bcfc203c4e196a25218e5fd80b)
BibTeX
@misc{dcd18ce1d47422,
title = {A Convergence Framework for Deep $V$-Learning: Error Propagation and Sharp Action-Gap Bounds — arXiv:2609.18782},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.18782},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d18ce1d47422; sha256 3a3eb174e826993f82e0004a71e88a788ccc80bcfc203c4e196a25218e5fd80b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.18782. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We establish convergence bounds for deep $V$-learning with horizon $H$. The algorithm fits a scalar value function to targets from executed transitions and selects actions using a predictive model and the value function. For current observed-successor targets with fresh true-kernel outcomes, the conditional mean is $\mathcal{T}^\beta V$, which averages over behavior-policy actions. The Bellman optimality update is $\mathcal{T} V$. We decompose the update error into six residuals: fitting, transition reuse, target construction, replay, action selection, and exploration. Under $L^s$ concentrability, their $L^p$ norms ($p=s/(s-1)$) control expected $L^1$ policy loss. The bound explicitly weights residuals from only the last $H-1$ update blocks, plus an initialization term for shorter runs. We quantify the cost of a shared sampling distribution across horizon levels. For statistical error bounds of order $n^{-\nu}$, we derive optimal continuous allocations and an integer allocation whose objective is within a factor $2^\nu$ of the constrained optimum. A margin condition with exponent $\alpha$ gives action error of order $\Lambda^{1+\alpha/p}$, where $\Lambda$ combines network drift and score error; a one-step construction proves the exponent sharp. Bounds on the distance between frozen and optimal scores transfer an optimal-gap condition to frozen-iterate gap bounds while retaining the mass of optimal ties. Survival probabilities and coverage conditions at deployment yield bounds for policies selected with approximate scores. Separate spatial ReLU networks per horizon level give a conditional neural regression rate, and the finite-state case gives a log-free expected fit rate. These results give expected policy-loss consistency for the fixed-horizon generative-reset approximate-ERM procedure with exact action scores and provide an explicit residual-decay criterion for FIFO/interleaved SGD.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Robotics (cs.RO); Optimization and Control (math.OC).
Pagina del lavoro: https://arxiv.org/abs/2609.18782 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T14:08:19.148Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).