Metraipolis › archive › dc_c36ec0d557c0
ZeroLock: Concurrent Memory-Efficient LLM Training via Modular Update Decoupling — arXiv:2608.07974
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.07974
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:52:46.019Z
- SHA-256 of the archived text
3bf7c7867b2e5ebf94f4863f5d9a9d411ef2fd5d4e67626f447a884bead62815- Stable URL of this page
https://metraipolis.com/documento/dc_c36ec0d557c0
How to cite
ZeroLock: Concurrent Memory-Efficient LLM Training via Modular Update Decoupling — arXiv:2608.07974 — arXiv (en), https://arxiv.org/abs/2608.07974 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c36ec0d557c0 (Metraipolis archive dc_c36ec0d557c0, sha256 3bf7c7867b2e5ebf94f4863f5d9a9d411ef2fd5d4e67626f447a884bead62815)
BibTeX
@misc{dcc36ec0d557c0,
title = {ZeroLock: Concurrent Memory-Efficient LLM Training via Modular Update Decoupling — arXiv:2608.07974},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.07974},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c36ec0d557c0; sha256 3bf7c7867b2e5ebf94f4863f5d9a9d411ef2fd5d4e67626f447a884bead62815},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.07974. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language model (LLM) fine-tuning at the edge adapts the model to scenario-specific data while preserving privacy. Although existing studies proposed pipeline parallelism to address the limited memory and computing resources of edge devices, they commonly rely on backpropagation (BP) training, which has a fundamental limitation of update locking and could experience severe throughput and memory bottlenecks. In this work, we propose a BP-free algorithm, called ZeroLock, that decouples the model updates into independent chunk updates by local objective construction. It breaks the update locking of BP and hence can improve throughput at the algorithm level and lower memory usage by reducing activation storage. To the best of our knowledge, we provide the first theoretical framework for such local objective construction-based approaches under general model chunk division by mapping local objectives to the global objective. We prove that ZeroLock has a convergence rate of $\tilde{\mathcal{O}}(1/\sqrt{T})$, which differs from BP only by polylogarithmic factors. We design a system for ZeroLock and build real-world prototypes, incorporating techniques such as early forwarding and failure recovery for efficient and robust implementation. Experiments on the prototype show that compared to BP-based baselines, ZeroLock reduces the memory by 26.5% and improves throughput by 4.9%.
Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-11 in cs.DC).
Categorie arXiv: Machine Learning (cs.LG) ; Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2608.07974 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:50:23.771Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).