Metraipolis › archive › dc_eb00e43b4677
Stochastic Gradient Methods with Online Scaling — arXiv:2609.11751
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.11751
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T06:01:07.728Z
- SHA-256 of the archived text
edda19c73781d74b9b5dfe94020c0bae125e4ff60681f03d91e2ac3814ead208- Stable URL of this page
https://metraipolis.com/documento/dc_eb00e43b4677
How to cite
Stochastic Gradient Methods with Online Scaling — arXiv:2609.11751 — arXiv (en), https://arxiv.org/abs/2609.11751 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_eb00e43b4677 (Metraipolis archive dc_eb00e43b4677, sha256 edda19c73781d74b9b5dfe94020c0bae125e4ff60681f03d91e2ac3814ead208)
BibTeX
@misc{dceb00e43b4677,
title = {Stochastic Gradient Methods with Online Scaling — arXiv:2609.11751},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.11751},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_eb00e43b4677; sha256 edda19c73781d74b9b5dfe94020c0bae125e4ff60681f03d91e2ac3814ead208},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11751. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
This paper introduces Stochastic Online Scaled Gradient Methods (SOSGM), a generalization of the recently developed adaptive preconditioning framework in arXiv:2505.23081 and arXiv:2509.11007 to stochastic optimization. Under standard assumptions, we establish convergence guarantees for SOSGM using large batchsize or variance reduction. SOSGM is compatible with popular diagonal and/or low-rank preconditioners as well as heavy-ball momentum, while maintaining memory and computation cost comparable to Adam. Extensive numerical experiments demonstrate the strong empirical performance of SOSGM. Using a diagonal preconditioner, SOSGM and its variants substantially outperform existing adaptive first-order methods across a range of statistical learning tasks. Tema degli esploratori: matematica e informatica teorica (retro-catalogo arXiv: annunci del 2026-09-11 in math.OC). Categorie arXiv: Optimization and Control (math.OC). Pagina del lavoro: https://arxiv.org/abs/2609.11751 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T05:52:41.421Z dalla pagina https://arxiv.org/catchup/math.OC/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).