Metraipolis › archive › dc_fb021b975117

Almost Sure Convergence Analysis of Stochastic Gradient Methods with Clipping and Additive Noise — arXiv:2609.12119

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.12119
License (as archived): CC0-1.0
Archived: 2026-10-16T03:15:31.543Z
SHA-256 of the archived text: 5c60554ef9d897c6e7b5440b87088d4c58afc92c1fc70587ddb88f4e9cb134e9
Stable URL of this page: https://metraipolis.com/documento/dc_fb021b975117

How to cite

Almost Sure Convergence Analysis of Stochastic Gradient Methods with Clipping and Additive Noise — arXiv:2609.12119 — arXiv (en), https://arxiv.org/abs/2609.12119 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fb021b975117 (Metraipolis archive dc_fb021b975117, sha256 5c60554ef9d897c6e7b5440b87088d4c58afc92c1fc70587ddb88f4e9cb134e9)

BibTeX

@misc{dcfb021b975117,
  title = {Almost Sure Convergence Analysis of Stochastic Gradient Methods with Clipping and Additive Noise — arXiv:2609.12119},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.12119},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fb021b975117; sha256 5c60554ef9d897c6e7b5440b87088d4c58afc92c1fc70587ddb88f4e9cb134e9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12119. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Stochastic gradient descent (SGD) with gradient clipping and additive noise has become a standard technique for training machine learning models, particularly in applications requiring robustness or privacy guarantees. However, clipping introduces a bias in stochastic gradients, while additive noise introduces additional variance, making the long-run behaviour of individual optimization trajectories difficult to characterize. In this work, we prove that SGD with clipping and additive Gaussian noise (SGD-CN) converges almost surely (a.s.) under smoothness and uniformly bounded stochastic-gradient noise assumptions, provided the step sizes satisfy some standard decaying conditions. Our analysis extends to momentum variants such as the stochastic heavy ball and Nesterov's accelerated gradient, where we show that careful energy constructions yield similar guarantees. These results provide stronger theoretical foundations for understanding the pathwise behaviour of clipped stochastic gradient methods and suggest that, despite the bias and noise introduced by clipping and perturbation, the algorithm remains stable in both convex and nonconvex regimes.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Optimization and Control (math.OC); Probability (math.PR).
Pagina del lavoro: https://arxiv.org/abs/2609.12119 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T03:05:44.016Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).