Metraipolis › archive › dc_1534573c98ef

One-shot Distributed Learning via Weighted Averaging with Server-side Information — arXiv:2611.04700

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.04700
License (as archived)
CC0-1.0
Archived
2026-11-04T21:52:55.678Z
SHA-256 of the archived text
be43bf931934af611b174e256d48547e1c5bfd2bbf76816fafacc518947352c9
Stable URL of this page
https://metraipolis.com/documento/dc_1534573c98ef

How to cite

One-shot Distributed Learning via Weighted Averaging with Server-side Information — arXiv:2611.04700 — arXiv (en), https://arxiv.org/abs/2611.04700 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1534573c98ef (Metraipolis archive dc_1534573c98ef, sha256 be43bf931934af611b174e256d48547e1c5bfd2bbf76816fafacc518947352c9)
BibTeX
@misc{dc1534573c98ef,
  title = {One-shot Distributed Learning via Weighted Averaging with Server-side Information — arXiv:2611.04700},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.04700},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1534573c98ef; sha256 be43bf931934af611b174e256d48547e1c5bfd2bbf76816fafacc518947352c9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04700. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Distributed learning has drawn considerable attention due to its capability of learning from distributed data without aggregating raw observations. However, frequent exchange of model updates can become the main bottleneck of communication. Therefore, one-shot approaches where each worker fits a local model and sends it to the server to aggregate emerge. It is known that averaging can achieve the so-called pooled-data mean squared error (MSE) order of a hypothetical node that has access to data from the server and all the workers, and performs empirical risk minimization (ERM) on it. However, when the number of workers increases too fast compared to the local sample size, nonzero local estimation bias prevents averaging from achieving this order. In this paper, we study whether independent server data can overcome this limitation. We consider $m$ workers with $n$ observations each and a server with $p$ observations, all i.i.d., giving $N=mn+p$ observations in total. Under certain regularity conditions, a pooled-data ERM achieves MSE of order $\mathcal O(N^{-1})$. We analyze deterministic nonnegative averaging of convex-loss M-estimators, with equal worker weights and the server's own estimator included. Under our assumptions, including nonzero local bias of order $n^{-\alpha}$, $\alpha\geq1$, sample-size weighted averaging achieves the pooled-data MSE order whenever $m=\mathcal O(n^{2\alpha-1})$. Above this worker scale, it does so if and only if $p=\Omega(nm\cdot m/n^{2\alpha-1})$. Hence, server-only estimation, whose MSE is of order $\mathcal{O}(p^{-1})$, can achieve pooled-data order while sample-size weighting fails. Analytical examples and synthetic experiments illustrate when averaging matches the pooled-data MSE order, complemented by CIFAR-10 experiments comparing aggregation strategies with linear and nonlinear classifiers.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (stat.ML) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.04700 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T21:38:49.821Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).