Metraipolis › archive › dc_da23d3d3c06d

Variance Reduction for Decentralized Stochastic Optimization with Multiple Local Updates — arXiv:2610.41927

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.41927
License (as archived)
CC0-1.0
Archived
2026-11-03T08:13:27.579Z
SHA-256 of the archived text
6484a7ec9d25ba81377ad390e22a238e3e6f5bb329ac0f7e573cd1515962090f
Stable URL of this page
https://metraipolis.com/documento/dc_da23d3d3c06d

How to cite

Variance Reduction for Decentralized Stochastic Optimization with Multiple Local Updates — arXiv:2610.41927 — arXiv (en), https://arxiv.org/abs/2610.41927 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_da23d3d3c06d (Metraipolis archive dc_da23d3d3c06d, sha256 6484a7ec9d25ba81377ad390e22a238e3e6f5bb329ac0f7e573cd1515962090f)
BibTeX
@misc{dcda23d3d3c06d,
  title = {Variance Reduction for Decentralized Stochastic Optimization with Multiple Local Updates — arXiv:2610.41927},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41927},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_da23d3d3c06d; sha256 6484a7ec9d25ba81377ad390e22a238e3e6f5bb329ac0f7e573cd1515962090f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41927. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multiple local updates are widely used in decentralized optimization to enhance communication efficiency, often in conjunction with stochastic gradients when exact gradient computation is costly. However, such algorithms generally converge only to a neighborhood of the optimum, whose size depends on the stepsize and the variance of gradient noise. Variance reduction has been incorporated into ADMM-based methods to achieve exact convergence in expectation, but their stepsizes typically scale inversely with both the square root of the network size and the cube of the number of local updates, leading to slow convergence as the network size or the number of local updates increases. In this paper, we propose a novel primal-dual-based decentralized optimization algorithm that integrates variance reduction with multiple local updates, with a stepsize independent of the network size and inversely proportional to the number of local updates (rather than its cube), thereby enabling significantly larger stepsizes and faster convergence. Moreover, to the best of our knowledge, we establish the first linear convergence guarantee in expectation for decentralized optimization with multiple local updates under stochastic gradients in the strongly convex setting. Beyond the theoretical guarantee, the proposed method exchanges only one variable per round, halving the communication cost compared with gradient-tracking-based methods. Numerical experiments further demonstrate its advantages in convergence speed and accuracy.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Optimization and Control (math.OC) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.41927 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T08:02:38.998Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).