Metraipolis › archive › dc_f70fd92da3f1
FedLore: Communication and Memory Efficient Federated Learning via Shared Gradient Low-Rank Projection — arXiv:2610.01620
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.01620
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T12:05:48.610Z
- SHA-256 of the archived text
42321e770185fd4536d4afcd31b80606e7f9cfb993e3d2f4780495bc91e57a25- Stable URL of this page
https://metraipolis.com/documento/dc_f70fd92da3f1
How to cite
FedLore: Communication and Memory Efficient Federated Learning via Shared Gradient Low-Rank Projection — arXiv:2610.01620 — arXiv (en), https://arxiv.org/abs/2610.01620 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f70fd92da3f1 (Metraipolis archive dc_f70fd92da3f1, sha256 42321e770185fd4536d4afcd31b80606e7f9cfb993e3d2f4780495bc91e57a25)
BibTeX
@misc{dcf70fd92da3f1,
title = {FedLore: Communication and Memory Efficient Federated Learning via Shared Gradient Low-Rank Projection — arXiv:2610.01620},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.01620},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f70fd92da3f1; sha256 42321e770185fd4536d4afcd31b80606e7f9cfb993e3d2f4780495bc91e57a25},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01620. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Federated training of foundation models is constrained by client memory and communication costs. LoRA-based methods reduce these costs through low-rank adapters, but their fixed rank budget can limit adaptation. Gradient low-rank optimization offers greater flexibility, yet independently chosen client subspaces create a problem we term \emph{subspace fragmentation}: local projections interact with data heterogeneity to bias aggregated directions, while aggregation can increase update rank and communication cost. Thus, accurate local gradient compression need not preserve global descent. We propose \texttt{FedLore}, which shares a low-rank optimization basis within each round and refreshes it across rounds. The shared basis enables exact aggregation in low-rank coordinates and eliminates the identified projection bias. Subspace refresh allows the accumulated model update to exceed the per-round rank budget. We characterize the aggregation bias and establish an $O(T^{-1/2})$ stationarity bound for the projected-SGD variant under a global-gradient coverage condition and standard smoothness and variance assumptions, with bounded gradient heterogeneity. Experiments on vision and language tasks, including federated pre-training, show that \texttt{FedLore} outperforms the evaluated low-rank adapter baselines and matches or exceeds full-parameter training, while reducing communication and optimizer-state memory.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.01620 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:27.967Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).