Metraipolis › archive › dc_6556ba2f4dbb

Replay in the Silent Degrees of Freedom: Continual Learning Without an Offline Phase — arXiv:2609.31630

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.31630
License (as archived)
CC0-1.0
Archived
2026-10-07T22:36:01.060Z
SHA-256 of the archived text
9e528b4dfe64a12aeb3dc069419658273f7a1acc0611fb6d41e06403d3653d42
Stable URL of this page
https://metraipolis.com/documento/dc_6556ba2f4dbb

How to cite

Replay in the Silent Degrees of Freedom: Continual Learning Without an Offline Phase — arXiv:2609.31630 — arXiv (en), https://arxiv.org/abs/2609.31630 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6556ba2f4dbb (Metraipolis archive dc_6556ba2f4dbb, sha256 9e528b4dfe64a12aeb3dc069419658273f7a1acc0611fb6d41e06403d3653d42)
BibTeX
@misc{dc6556ba2f4dbb,
  title = {Replay in the Silent Degrees of Freedom: Continual Learning Without an Offline Phase — arXiv:2609.31630},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31630},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6556ba2f4dbb; sha256 9e528b4dfe64a12aeb3dc069419658273f7a1acc0611fb6d41e06403d3653d42},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31630. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Replay-based continual learning rehearses past data either in an offline phase, during which the agent stops acting, or interleaved with the live stream, where it perturbs the computation serving the current input. An agent that learns in deployment can afford neither. Motivated by local sleep, the use-dependent off periods of individual cortical circuits in awake animals, we show that replay can instead be written into the degrees of freedom the current input leaves unused. In a network with k-winner-take-all hidden layers, confining replay updates to synapses whose presynaptic unit is silent or whose postsynaptic unit is inactive leaves the hidden computation on the current batch invariant: exactly so for silent and suppressed units, and for all but 0.3% of samples in practice. A refractory rule under which units that have just fired sit out the next competition doubles the width of this channel and carries most of the accuracy. On class-incremental split-MNIST the resulting learner, with no offline phase, matches or exceeds the best offline rehearsal schedule and outperforms experience replay, ER-ACE and unmasked interleaved replay, each re-tuned under the same micro-batch schedule. Against DER++ the comparison splits by protocol: with five epochs per task DER++ leads by 1.5 points once it runs under that schedule, and in a single pass over the stream, the regime closest to the agent deployment setting, the system leads it by 1.6 while offline rehearsal falls 15 points behind. On split CIFAR-10 it again leads offline rehearsal and experience replay, but trails ER-ACE and DER++ by two to three points. The construction is not tied to the local learner: on a backprop network with k-winner-take-all hidden layers under the same schedule, refractory rotation adds half a point in five epochs and three in a single pass, and isolation again costs nothing on top of it.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.31630 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.503Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).