Metraipolis › archive › dc_ea90cb6386b5

Learning Under Forgetting: Statistical Support-Selective Retention in Stochastic Training Dynamics — arXiv:2609.38768

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.38768
License (as archived)
CC0-1.0
Archived
2026-10-07T12:44:13.647Z
SHA-256 of the archived text
2395e3704149812bba2a7949f5b2dbafef135966ed20df264e0e78ba30c13cbc
Stable URL of this page
https://metraipolis.com/documento/dc_ea90cb6386b5

How to cite

Learning Under Forgetting: Statistical Support-Selective Retention in Stochastic Training Dynamics — arXiv:2609.38768 — arXiv (en), https://arxiv.org/abs/2609.38768 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ea90cb6386b5 (Metraipolis archive dc_ea90cb6386b5, sha256 2395e3704149812bba2a7949f5b2dbafef135966ed20df264e0e78ba30c13cbc)
BibTeX
@misc{dcea90cb6386b5,
  title = {Learning Under Forgetting: Statistical Support-Selective Retention in Stochastic Training Dynamics — arXiv:2609.38768},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.38768},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ea90cb6386b5; sha256 2395e3704149812bba2a7949f5b2dbafef135966ed20df264e0e78ba30c13cbc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38768. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Prior work has shown that neural networks exhibit implicit biases toward low-complexity structure (e.g., spectral bias), memorization dynamics, and compression-like effects during training, but a unified dynamical account of selective retention remains incomplete. We propose Repeated Reinforcement with Persistent Forgetting (RPF) dynamics, a minimal framework in which repeated exposure reinforces patterns and structures that recur in the data, while persistent forgetting attenuates learned information. This view treats forgetting not merely as a failure mode, but as a selection mechanism. We build the theory in three successive layers. First, in an independent-feature model, we derive an exposure-selective survival law and a support-dependent retention boundary characterizing which patterns persist under forgetting. Second, in a shared-parameter model, we show that forgetting induces spectral filtering over covariance modes, preserving strongly supported shared components while suppressing weak ones. Third, under small-step and norm/coding approximations, we show how RPF dynamics induce an implicit trade-off between data fitting and the cost of stored information, yielding Minimum Description Length (MDL)-like compression. Controlled experiments provide evidence for this reinforcement--forgetting selection mechanism in scalar memories and a nonlinear shared network. Joint reinforcement and attenuation interventions shift conditional retention, while matched exposure counts reveal forgetting-dependent effects of reinforcement timing and changes in the composition of the retained set. Together, these results show that repeated reinforcement and persistent forgetting jointly provide a controllable source of inductive bias beyond neural architecture and scale.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.38768 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:13.936Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).