Metraipolis › archive › dc_62e40df5156e
RASP: Read-All, Sparse-Plasticity Continual Learning in a Stationary-Size LoRA — arXiv:2611.03513
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.03513
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:23:20.337Z
- SHA-256 of the archived text
8640bdb864647c0d78bc1621e357c7271b6874f5af5fe87452c75a6c4acb23c7- Stable URL of this page
https://metraipolis.com/documento/dc_62e40df5156e
How to cite
RASP: Read-All, Sparse-Plasticity Continual Learning in a Stationary-Size LoRA — arXiv:2611.03513 — arXiv (en), https://arxiv.org/abs/2611.03513 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_62e40df5156e (Metraipolis archive dc_62e40df5156e, sha256 8640bdb864647c0d78bc1621e357c7271b6874f5af5fe87452c75a6c4acb23c7)
BibTeX
@misc{dc62e40df5156e,
title = {RASP: Read-All, Sparse-Plasticity Continual Learning in a Stationary-Size LoRA — arXiv:2611.03513},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.03513},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_62e40df5156e; sha256 8640bdb864647c0d78bc1621e357c7271b6874f5af5fe87452c75a6c4acb23c7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03513. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Parameter-efficient continual learning on pre-trained Vision Transformers largely relies on a costly premise: preventing catastrophic forgetting requires either spawning a dedicated LoRA branch per session---causing adapter parameters to scale linearly with task horizon---or applying sparse Mixture-of-Experts gating symmetrically to both forward and backward passes, which sacrifices representational rank at test time. We argue that reading consolidated knowledge and writing new task credit are fundamentally asymmetric operations. Because visual features in a shared low-rank space distribute across rank-1 atoms rather than localizing in disjoint task silos, forward inference should read all atoms at full rank capacity, whereas backward optimization should restrict plasticity to a sparse subset of input-aligned channels. We introduce RASP (Read-All, Sparse-Plasticity LoRA), which adapts a single stationary-size adapter across sequential tasks. At each step, a scene-modulated cosine router selects the top-$q$ unlocked atoms to receive gradients, blocks updates on the remaining $r-q$ channels, and penalizes Frobenius Gram coherence between active and non-updated output directions. Across sessions, the router's empirical activation trace and each atom's exact spectral norm form a zero-overhead geometric consolidation index $\omega_j = \sqrt{\tilde{c}_j \tilde{\sigma}_j}$ that drives Cauchy step attenuation and boundary anchor locking. Across CIFAR-100, CUB-200, ImageNet-R, and ImageNet-A, RASP matches or surpasses branch-expanding LoRA learners under a strictly constant parameter footprint.
Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.03513 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T11:02:37.716Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).