Metraipolis › archive › dc_643fd0df6517

Optimal Two-Step Stepsize Schedule for Stochastic Gradient Methods — arXiv:2608.15035

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.15035
License (as archived)
CC0-1.0
Archived
2026-10-18T20:18:09.353Z
SHA-256 of the archived text
f78f339f1fc87177e9c5e1f9b724071bb0df4fa1c640a1ca9c8ad1a836ef7fb1
Stable URL of this page
https://metraipolis.com/documento/dc_643fd0df6517

How to cite

Optimal Two-Step Stepsize Schedule for Stochastic Gradient Methods — arXiv:2608.15035 — arXiv (en), https://arxiv.org/abs/2608.15035 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_643fd0df6517 (Metraipolis archive dc_643fd0df6517, sha256 f78f339f1fc87177e9c5e1f9b724071bb0df4fa1c640a1ca9c8ad1a836ef7fb1)
BibTeX
@misc{dc643fd0df6517,
  title = {Optimal Two-Step Stepsize Schedule for Stochastic Gradient Methods — arXiv:2608.15035},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.15035},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_643fd0df6517; sha256 f78f339f1fc87177e9c5e1f9b724071bb0df4fa1c640a1ca9c8ad1a836ef7fb1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15035. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Structured nonconstant large stepsizes can improve the convergence of gradient descent in the deterministic setting. However, in stochastic optimization, aggressive stepsizes can amplify oracle noise and hinder the convergence of stochastic gradient methods. We characterize the globally optimal two-step stepsize schedule for stochastic gradient methods applied to strongly convex and smooth functions, assuming access only to unbiased stochastic gradient estimates with finite support and bounded variance. The optimal schedule depends on the ratio of the initial optimality gap to the noise level and exhibits several distinct regimes. As the influence of stochastic noise diminishes, the optimal two-step stepsizes become larger, reflecting a balance between the benefits of faster iterate convergence and the perturbations induced by stochastic noise.

Tema degli esploratori: matematica e informatica teorica (retro-catalogo arXiv: annunci del 2026-08-18 in math.OC).
Categorie arXiv: Optimization and Control (math.OC).
Pagina del lavoro: https://arxiv.org/abs/2608.15035 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T19:45:07.835Z dalla pagina https://arxiv.org/catchup/math.OC/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).