Metraipolis › archive › dc_c92289cbfe2f
Long-Time Behavior of Zero-Noise Mean-Field Adam — arXiv:2611.01658
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.01658
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:31:46.194Z
- SHA-256 of the archived text
d29b9eaebe6954cd881e3e819fd63ce409d6cc99e31213d85d95ae0f043cd4df- Stable URL of this page
https://metraipolis.com/documento/dc_c92289cbfe2f
How to cite
Long-Time Behavior of Zero-Noise Mean-Field Adam — arXiv:2611.01658 — arXiv (en), https://arxiv.org/abs/2611.01658 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c92289cbfe2f (Metraipolis archive dc_c92289cbfe2f, sha256 d29b9eaebe6954cd881e3e819fd63ce409d6cc99e31213d85d95ae0f043cd4df)
BibTeX
@misc{dcc92289cbfe2f,
title = {Long-Time Behavior of Zero-Noise Mean-Field Adam — arXiv:2611.01658},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.01658},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c92289cbfe2f; sha256 d29b9eaebe6954cd881e3e819fd63ce409d6cc99e31213d85d95ae0f043cd4df},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01658. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Adaptive methods such as Adam are the most commonly used methods for training neural networks, but their long-time behavior is not as well understood as that of non-adaptive methods. This is particularly true in the mean-field limit, where the dynamics of infinitely many neurons are described as a distribution over parameters which satisfies a self-consistency condition. In this paper, we study a deterministic continuous-time model of Adam in the mean-field setting. Under a condition relating the averaging rates of the momentum and second-moment variables, the size of the gradients, and a regularization parameter, we prove that the distribution of parameters converges to the set of equilibria. This condition extends the range known for finite-dimensional models, and we show that it is sharp. Under an additional Lojasiewicz-type inequality for distributions, we prove that individual parameter trajectories converge and obtain quantitative rates for this convergence. We verify the qualitative convergence assumptions for a regularized two-layer neural network. We also show that adding noise to the momentum prevents the system from approaching these equilibria and rules out many free-energy type Lyapunov functionals. Finally, numerical experiments illustrate the energy estimates, give evidence of persistent oscillations outside the sufficient parameter range, and compare the observed decay with the predicted rates. Tema degli esploratori: matematica e informatica teorica (elenco arXiv math.OC, nuovi lavori del giorno). Categorie arXiv: Optimization and Control (math.OC). Pagina del lavoro: https://arxiv.org/abs/2611.01658 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T11:04:22.241Z dalla pagina https://arxiv.org/list/math.OC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).