Metraipolis › archive › dc_49498c37032e
RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks — arXiv:2609.00078
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.00078
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T06:32:47.159Z
- SHA-256 of the archived text
93cbc4583f94c695824c487d1a8b13f4bf980ca5f170d0f4a3ebc6a0968e5a0d- Stable URL of this page
https://metraipolis.com/documento/dc_49498c37032e
How to cite
RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks — arXiv:2609.00078 — arXiv (en), https://arxiv.org/abs/2609.00078 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_49498c37032e (Metraipolis archive dc_49498c37032e, sha256 93cbc4583f94c695824c487d1a8b13f4bf980ca5f170d0f4a3ebc6a0968e5a0d)
BibTeX
@misc{dc49498c37032e,
title = {RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks — arXiv:2609.00078},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.00078},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_49498c37032e; sha256 93cbc4583f94c695824c487d1a8b13f4bf980ca5f170d0f4a3ebc6a0968e5a0d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.00078. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Parameter-efficient fine-tuning methods such as LoRA have become a standard approach for adapting large foundation models. Adopting fine-tuning to distributed settings faces several challenges. Most existing distributed LoRA methods rely on centralized aggregation, and gossip-based decentralized LoRA requires repeated synchronization among multiple model copies. Both methods incur significant communication overhead and introduce errors due to simultaneous aggregation of multiple model updates. In this paper, we take a different perspective and propose a random-walk-based LoRA fine-tuning scheme. Instead of maintaining multiple model replicas, a single model token traverses the network and is updated sequentially using local fine-tuning objectives. This design eliminates the need for global synchronization, substantially reduces communication and computation costs, and avoids aggregation errors. We provide rigorous convergence guarantees for non-convex objectives under standard assumptions. Through empirical results on multiple NLP tasks and graph topologies, we show that the proposed method achieves competitive task performance with substantially less communication and computation than gossip-based LoRA. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.00078 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T06:15:12.736Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).