Metraipolis › archive › dc_67e451bcd05b

Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning — arXiv:2610.11076

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.11076
License (as archived)
CC0-1.0
Archived
2026-10-09T05:57:01.454Z
SHA-256 of the archived text
23c0075a0d2850f3a6058558c598fd1da82be665769d4f8491e086431c12d13e
Stable URL of this page
https://metraipolis.com/documento/dc_67e451bcd05b

How to cite

Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning — arXiv:2610.11076 — arXiv (en), https://arxiv.org/abs/2610.11076 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_67e451bcd05b (Metraipolis archive dc_67e451bcd05b, sha256 23c0075a0d2850f3a6058558c598fd1da82be665769d4f8491e086431c12d13e)
BibTeX
@misc{dc67e451bcd05b,
  title = {Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning — arXiv:2610.11076},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.11076},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_67e451bcd05b; sha256 23c0075a0d2850f3a6058558c598fd1da82be665769d4f8491e086431c12d13e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.11076. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Domain-specific continual adaptation of LLMs risks catastrophic forgetting, creating a fundamental tension between acquiring new capabilities and preserving those learned during pretraining. PEFT mitigates this problem by restricting the number of trainable parameters, but existing methods lack a principled unit for deciding where plasticity should be allocated and stability should be preserved. We identify the singular-vector channel as a natural unit for managing this trade-off. Each channel represents an input-output transformation, which can be updated to acquire new knowledge or fixed to preserve pretrained capabilities. Based on this perspective, we introduce SVC, a parameter-efficient continual-learning method that selectively updates Singular-Vector Channels. Before fine-tuning, SVC uses domain-specific data to estimate each channel's adaptation benefit and a fixed public general-domain corpus only as a history activation proxy for estimating forgetting cost. It then adaptively selects trainable channels based on these scores via knee-based cost screening, Pareto-front filtering, and Otsu thresholding. Experimental results across four LLM families and eight downstream tasks show that SVC better preserves pretrained capabilities while achieving strong downstream performance relative to existing PEFT baselines. Further analysis of channel scoring and selection demonstrates that selective plasticity at the singular-vector-channel level enables effective continual LLM adaptation.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.11076 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:47:01.502Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).