Metraipolis › archive › dc_dbbb79114a9c

CommuteProp: Decoupled Training for Communication Bound Split LLM Fine-Tuning — arXiv:2610.05105

Source
arXiv
Original
https://arxiv.org/abs/2610.05105
License (as archived)
CC0-1.0
Archived
2026-10-06T05:29:00.941Z
SHA-256 of the archived text
18bb68591d07fb0e3792bd0bcda52690806b5aae7156a9799c88fc7c838fd2c7
Stable URL of this page
https://metraipolis.com/documento/dc_dbbb79114a9c

How to cite

CommuteProp: Decoupled Training for Communication Bound Split LLM Fine-Tuning — arXiv:2610.05105 — arXiv, https://arxiv.org/abs/2610.05105 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dbbb79114a9c (Metraipolis archive dc_dbbb79114a9c, sha256 18bb68591d07fb0e3792bd0bcda52690806b5aae7156a9799c88fc7c838fd2c7)
BibTeX
@misc{dcdbbb79114a9c,
  title = {CommuteProp: Decoupled Training for Communication Bound Split LLM Fine-Tuning — arXiv:2610.05105},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.05105},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dbbb79114a9c; sha256 18bb68591d07fb0e3792bd0bcda52690806b5aae7156a9799c88fc7c838fd2c7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05105. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Split learning has emerged as a promising paradigm for privacy-preserving LLM fine-tuning, yet its practical deployment is severely hindered by the sequential communication-computation bottleneck. In conventional synchronous pipelines, clients remain idle while waiting for server-side gradients, resulting in substantial training inefficiency. We propose CommuteProp, an asynchronous split-learning algorithm that decouples the training process into two concurrent phases: a cross-block forward-backward pass and an in-block weight update. By overlapping computation with communication, CommuteProp reduces the marginal cycle time from a sum of all stage latencies to the dominant computational bottleneck. We provide a rigorous asynchronous error and convergence analysis. Moreover, we derive an NS preconditioner method based on our analysis to further mitigate staleness-induced noise. Comprehensive experiments indicate that our algorithm achieves substantial throughput gains while maintaining accuracy comparable to synchronous methods; furthermore, it functions as a plug-and-play module that not only accommodates but actively enhances existing privacy enhancement methods.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2610.05105 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:35.302Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).