Metraipolis › archive › dc_dbbb79114a9c
CommuteProp: Decoupled Training for Communication Bound Split LLM Fine-Tuning — arXiv:2610.05105
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.05105
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:29:00.941Z
- SHA-256 of the archived text
18bb68591d07fb0e3792bd0bcda52690806b5aae7156a9799c88fc7c838fd2c7- Stable URL of this page
https://metraipolis.com/documento/dc_dbbb79114a9c
How to cite
CommuteProp: Decoupled Training for Communication Bound Split LLM Fine-Tuning — arXiv:2610.05105 — arXiv, https://arxiv.org/abs/2610.05105 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dbbb79114a9c (Metraipolis archive dc_dbbb79114a9c, sha256 18bb68591d07fb0e3792bd0bcda52690806b5aae7156a9799c88fc7c838fd2c7)
BibTeX
@misc{dcdbbb79114a9c,
title = {CommuteProp: Decoupled Training for Communication Bound Split LLM Fine-Tuning — arXiv:2610.05105},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.05105},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dbbb79114a9c; sha256 18bb68591d07fb0e3792bd0bcda52690806b5aae7156a9799c88fc7c838fd2c7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05105. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Split learning has emerged as a promising paradigm for privacy-preserving LLM fine-tuning, yet its practical deployment is severely hindered by the sequential communication-computation bottleneck. In conventional synchronous pipelines, clients remain idle while waiting for server-side gradients, resulting in substantial training inefficiency. We propose CommuteProp, an asynchronous split-learning algorithm that decouples the training process into two concurrent phases: a cross-block forward-backward pass and an in-block weight update. By overlapping computation with communication, CommuteProp reduces the marginal cycle time from a sum of all stage latencies to the dominant computational bottleneck. We provide a rigorous asynchronous error and convergence analysis. Moreover, we derive an NS preconditioner method based on our analysis to further mitigate staleness-induced noise. Comprehensive experiments indicate that our algorithm achieves substantial throughput gains while maintaining accuracy comparable to synchronous methods; furthermore, it functions as a plug-and-play module that not only accommodates but actively enhances existing privacy enhancement methods. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2610.05105 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:35.302Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).