Metraipolis › archive › dc_fc57495474af
ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs — arXiv:2610.00431
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.00431
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T15:32:39.507Z
- SHA-256 of the archived text
f6735e07bf39f98df47ebdee9c7478f890fbfdd43f57ec6c63456432c4c00e88- Stable URL of this page
https://metraipolis.com/documento/dc_fc57495474af
How to cite
ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs — arXiv:2610.00431 — arXiv (en), https://arxiv.org/abs/2610.00431 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fc57495474af (Metraipolis archive dc_fc57495474af, sha256 f6735e07bf39f98df47ebdee9c7478f890fbfdd43f57ec6c63456432c4c00e88)
BibTeX
@misc{dcfc57495474af,
title = {ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs — arXiv:2610.00431},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.00431},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fc57495474af; sha256 f6735e07bf39f98df47ebdee9c7478f890fbfdd43f57ec6c63456432c4c00e88},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.00431. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Continual parameter-efficient fine-tuning for large language models (LLMs) must balance retention of previously acquired knowledge, adaptation to new tasks, and strict parameter budgets. We present \textbf{ChainLoRA}, a replay-free continual merging framework built on chain-updated task-vector geometry. From a parameter-merging perspective, we formulate a geometric view of forgetting through a measurable interaction between task updates, separating directional overlap from coefficient coupling. Building on this view, ChainLoRA combines chain-updated training with post-stream adaptive SVD merging. During training, initialization and a one-sided orthogonality proxy use only the last carrier, keeping their historical-state footprint and regularization overhead constant as the task stream grows. At merging time, Adaptive SVD extracts a shared carrier and aligns it to the latest task through Procrustes adaptation. Our theoretical analysis shows that Procrustes adaptation facilitates geometric approximate separation of shared and task-specific components. The one-sided proxy further bounds inter-task interference. An effective-rank penalty additionally promotes efficient utilization of the task subspace during continual learning. Experiments show that ChainLoRA achieves state-of-the-art performance among the evaluated replay-free methods on the Large and SuperNI benchmarks, while remaining competitive on Standard CL and attaining almost the closest average scores to the evaluated replay-based method across all three benchmarks.
Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-10-02 in stat.ML).
Categorie arXiv: Machine Learning (stat.ML) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.00431 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T15:12:55.863Z dalla pagina https://arxiv.org/catchup/stat.ML/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).