Metraipolis › archive › dc_91db65b47803
Variance Reduction for Decentralized Stochastic Optimization with Multiple Local Updates — arXiv:2610.41927
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.41927
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-02T04:53:27.952Z
- SHA-256 of the archived text
4ff7a3aa02a809f45021682a7ac07a19c813b98910068c016374506f08cce867- Stable URL of this page
https://metraipolis.com/documento/dc_91db65b47803
How to cite
Variance Reduction for Decentralized Stochastic Optimization with Multiple Local Updates — arXiv:2610.41927 — arXiv (en), https://arxiv.org/abs/2610.41927 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_91db65b47803 (Metraipolis archive dc_91db65b47803, sha256 4ff7a3aa02a809f45021682a7ac07a19c813b98910068c016374506f08cce867)
BibTeX
@misc{dc91db65b47803,
title = {Variance Reduction for Decentralized Stochastic Optimization with Multiple Local Updates — arXiv:2610.41927},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.41927},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_91db65b47803; sha256 4ff7a3aa02a809f45021682a7ac07a19c813b98910068c016374506f08cce867},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41927. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multiple local updates are widely used in decentralized optimization to enhance communication efficiency, often in conjunction with stochastic gradients when exact gradient computation is costly. However, such algorithms generally converge only to a neighborhood of the optimum, whose size depends on the stepsize and the variance of gradient noise. Variance reduction has been incorporated into ADMM-based methods to achieve exact convergence in expectation, but their stepsizes typically scale inversely with both the square root of the network size and the cube of the number of local updates, leading to slow convergence as the network size or the number of local updates increases. In this paper, we propose a novel primal-dual-based decentralized optimization algorithm that integrates variance reduction with multiple local updates, with a stepsize independent of the network size and inversely proportional to the number of local updates (rather than its cube), thereby enabling significantly larger stepsizes and faster convergence. Moreover, to the best of our knowledge, we establish the first linear convergence guarantee in expectation for decentralized optimization with multiple local updates under stochastic gradients in the strongly convex setting. Beyond the theoretical guarantee, the proposed method exchanges only one variable per round, halving the communication cost compared with gradient-tracking-based methods. Numerical experiments further demonstrate its advantages in convergence speed and accuracy. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Optimization and Control (math.OC) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.41927 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-02T04:45:12.967Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).