Metraipolis › archive › dc_a0bd2c397301

Sustaining Plasticity via Learnable Wavelet Activations in Continual Learning — arXiv:2608.12874

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.12874
License (as archived)
CC0-1.0
Archived
2026-10-19T04:27:27.031Z
SHA-256 of the archived text
33b2e5e848f28eb4ee548b8a522199dc26f78aae077030c899b3205ca1c88355
Stable URL of this page
https://metraipolis.com/documento/dc_a0bd2c397301

How to cite

Sustaining Plasticity via Learnable Wavelet Activations in Continual Learning — arXiv:2608.12874 — arXiv (en), https://arxiv.org/abs/2608.12874 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a0bd2c397301 (Metraipolis archive dc_a0bd2c397301, sha256 33b2e5e848f28eb4ee548b8a522199dc26f78aae077030c899b3205ca1c88355)
BibTeX
@misc{dca0bd2c397301,
  title = {Sustaining Plasticity via Learnable Wavelet Activations in Continual Learning — arXiv:2608.12874},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.12874},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a0bd2c397301; sha256 33b2e5e848f28eb4ee548b8a522199dc26f78aae077030c899b3205ca1c88355},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.12874. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Plasticity loss has emerged as a critical challenge in continual learning that significantly hinders the acquisition of sequential tasks. While optimizing activation designs offers a potential solution, current fixed-form functions suffer from an inherent spectral bias towards low-frequency variations, whereas learnable variants permit unconstrained updates that induce catastrophic forgetting. To address these limitations, we propose a novel learnable wavelet activation that decomposes the activation function into low-frequency and high-frequency components to explicitly counter spectral bias. Furthermore, we employ dynamic wavelet injection to adaptively enhance plasticity for new tasks, alongside a regularization strategy to ensure the stability of previous learned knowledge. Theoretically, we provide rigorous mathematical guarantees for the proposed framework, proving the structural necessity of the hybrid wavelet architecture for efficient $L^2$ approximation and demonstrating that the decoupled learning rate mechanism successfully restores network plasticity for high-frequency information. Additionally, we provide a formal derivation of the loss-driven injection trigger mechanism to precisely guide the injection. Extensive empirical evaluations demonstrate that our approach maintains superior trainability and generalization throughout the learning process and achieves state-of-the-art performance across diverse continual learning benchmarks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-14 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.12874 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T04:20:35.829Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).