Metraipolis › archive › dc_57fd87e71880
Exact Tail Laws for Nonlinear SGD with Infinite-Variance Gradients — arXiv:2610.23750
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.23750
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T09:37:31.887Z
- SHA-256 of the archived text
8a917066c31be02d2ba3321b3a08116cd1ea37d25e12ba5537f6d98837434c8c- Stable URL of this page
https://metraipolis.com/documento/dc_57fd87e71880
How to cite
Exact Tail Laws for Nonlinear SGD with Infinite-Variance Gradients — arXiv:2610.23750 — arXiv (en), https://arxiv.org/abs/2610.23750 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_57fd87e71880 (Metraipolis archive dc_57fd87e71880, sha256 8a917066c31be02d2ba3321b3a08116cd1ea37d25e12ba5537f6d98837434c8c)
BibTeX
@misc{dc57fd87e71880,
title = {Exact Tail Laws for Nonlinear SGD with Infinite-Variance Gradients — arXiv:2610.23750},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.23750},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_57fd87e71880; sha256 8a917066c31be02d2ba3321b3a08116cd1ea37d25e12ba5537f6d98837434c8c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.23750. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
With infinite-variance gradient noise, SGD can have infinite expected excess objective even when most iterates are close to the optimum. We characterize its accuracy through exact first-order tail probabilities for the last iterate of nonlinear, constant-stepsize SGD. For regularly varying innovations of index $t\in(1,2)$, the leading measure transports one large gradient shock through the local Hessian flow. This is the same measure that governs the central stable Ornstein--Uhlenbeck limit; we prove its relative accuracy at shrinking tolerances above the central scale, with state-dependent noise and a growing observation horizon. The admissible window is nonempty for every available gradient moment $p>1$. The main technical step is a moment bound at the central fluctuation scale and a lower-moment control of the nonlinear error. The result yields explicit projection and excess-objective tails, extreme quantiles, and a factor $b^{1-t}$ for fixed batch size $b$. For affine population gradients, a sharper comparison gives the same law at fixed tolerances, including random sample Hessians. Coupled simulations in dimensions one to one hundred examine relative calibration, objective quantiles, batch effects, and the error due to random curvature.
Tema degli esploratori: statistica e scienza dei dati (elenco arXiv stat.ML, nuovi lavori del giorno).
Categorie arXiv: Statistics Theory (math.ST) ; Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2610.23750 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T09:32:44.742Z dalla pagina https://arxiv.org/list/stat.ML/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).