Metraipolis › archive › dc_7dadcdd1698c
When the Merge Coefficient Stops Mattering: Proximity Regularized Merging for Continual LoRA Adaptation — arXiv:2609.32332
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.32332
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T22:36:30.641Z
- SHA-256 of the archived text
32edf8cbdbfedd6a881490de455228655476063f40969b5975968b45c7d75ad4- Stable URL of this page
https://metraipolis.com/documento/dc_7dadcdd1698c
How to cite
When the Merge Coefficient Stops Mattering: Proximity Regularized Merging for Continual LoRA Adaptation — arXiv:2609.32332 — arXiv (en), https://arxiv.org/abs/2609.32332 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7dadcdd1698c (Metraipolis archive dc_7dadcdd1698c, sha256 32edf8cbdbfedd6a881490de455228655476063f40969b5975968b45c7d75ad4)
BibTeX
@misc{dc7dadcdd1698c,
title = {When the Merge Coefficient Stops Mattering: Proximity Regularized Merging for Continual LoRA Adaptation — arXiv:2609.32332},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.32332},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7dadcdd1698c; sha256 32edf8cbdbfedd6a881490de455228655476063f40969b5975968b45c7d75ad4},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32332. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Rehearsal-free continual learning with parameter-efficient adapters can be cast as a sequence of task-vector write-in operations: for each new task, a low-rank adapter is learned and merged into a running model. We propose Proximity Regularized Merging (PRM), a minimal modification to sequential LoRA merging that adds a proximal penalty during task-vector training without changing the subsequent write-in rule. PRM acts as a robust task-vector regularizer: in the reported Base->+Prox diagnostics, it improves AAA across multiple write-in rules, backbones, and class-incremental settings, while its fixed-coefficient variant remains competitive with strong coefficient-based baselines. Mechanistically, matched-prefix norm controls and proximal-strength sweeps show that proximal training shrinks the task-vector radius, lowers Fisher-weighted interference, broadens the coefficient plateau, and exposes a stability-plasticity trade-off. Together, these results suggest that the effectiveness of sequential LoRA merging depends not only on how much of a task vector is written in, but also on whether the task vector itself has been trained to be mergeable. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.32332 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T22:14:27.661Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).