Metraipolis › archive › dc_a41b2b6e4e2d

Stationary Bias and Extrapolation in Nonlinear Two-Timescale Stochastic Approximation — arXiv:2610.10246

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.10246
License (as archived): CC0-1.0
Archived: 2026-10-08T11:11:57.098Z
SHA-256 of the archived text: fa94362658479364d98cd19dfa7f4db3129d4e374aae666f35e5b6d1193ed515
Stable URL of this page: https://metraipolis.com/documento/dc_a41b2b6e4e2d

How to cite

Stationary Bias and Extrapolation in Nonlinear Two-Timescale Stochastic Approximation — arXiv:2610.10246 — arXiv (en), https://arxiv.org/abs/2610.10246 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a41b2b6e4e2d (Metraipolis archive dc_a41b2b6e4e2d, sha256 fa94362658479364d98cd19dfa7f4db3129d4e374aae666f35e5b6d1193ed515)

BibTeX

@misc{dca41b2b6e4e2d,
  title = {Stationary Bias and Extrapolation in Nonlinear Two-Timescale Stochastic Approximation — arXiv:2610.10246},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.10246},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a41b2b6e4e2d; sha256 fa94362658479364d98cd19dfa7f4db3129d4e374aae666f35e5b6d1193ed515},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.10246. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Constant-step stochastic approximation generally has a nonzero stationary mean error that persists under time averaging. This paper studies that error for nonlinear two-timescale recursions driven by an exogenous finite-state Markov chain. Under stated smoothness assumptions and conditions on the stationary distribution, we derive a first-order bias expansion whose error bound remains uniform as the slow step size becomes much smaller than the fast step size. Fast-manifold coordinates keep the associated covariance equation regular in this limit. For fast step $\eta$ and slow step $\varepsilon$, the expansion reveals a mixed contribution $\varepsilon^2/\eta$ alongside terms linear in each step size. This dependence matters for bias reduction: along power-law step-size paths, the bias exponents need not be integers, so Richardson--Romberg extrapolation requires weights matched to the path. An exactly solvable nonlinear Markov example verifies the coefficients. We verify localization for temporal-difference learning and compare finite-run extrapolation at equal update budgets. For finite runs, we bound the initialization error of tail averages on both timescales under an additional coupling assumption. In the special case of additive independent noise, signed third-moment cancellation yields a sharper remainder.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.10246 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T10:14:06.166Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).