Metraipolis › archive › dc_32460ea9832f
How Temporal Correlations Shape Memory in Linear Recurrent Neural Networks — arXiv:2609.00420
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.00420
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T06:52:03.482Z
- SHA-256 of the archived text
eeeff3a4963ca4c0b9512a03285d4b4535c9719875122278268a622e42de3d48- Stable URL of this page
https://metraipolis.com/documento/dc_32460ea9832f
How to cite
How Temporal Correlations Shape Memory in Linear Recurrent Neural Networks — arXiv:2609.00420 — arXiv (en), https://arxiv.org/abs/2609.00420 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_32460ea9832f (Metraipolis archive dc_32460ea9832f, sha256 eeeff3a4963ca4c0b9512a03285d4b4535c9719875122278268a622e42de3d48)
BibTeX
@misc{dc32460ea9832f,
title = {How Temporal Correlations Shape Memory in Linear Recurrent Neural Networks — arXiv:2609.00420},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.00420},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_32460ea9832f; sha256 eeeff3a4963ca4c0b9512a03285d4b4535c9719875122278268a622e42de3d48},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.00420. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The linear recurrent neural network (LRNN) is a simple model for studying how much memory a network builds up as it trains. For uncorrelated inputs, earlier work found that training itself settles the network between keeping the past and reacting only to the present. Real sequences are correlated, and we solve the learning dynamics exactly for correlated inputs. In the solution, keeping the past carries a cost. The whole effect of correlation lands on that cost. This cost reduces to the earlier one when inputs are uncorrelated and grows once they are positively correlated. Three findings follow. (1) Correlation reshapes the course of learning, not only its end. Memory builds, overshoots, and is partly removed, and the settled network keeps less of the past. (2) Memory switches off at a threshold set by one number, how much each input resembles the one just before it. Neither sequence length nor longer-range correlation moves this threshold. Memory is worth keeping only when the task needs the previous input more than the current input already supplies it through correlation with the past. (3) The best network changes too. Zero error demands a feedthrough, a path that passes the current input straight to the network's output and remembers nothing, and training builds it unprompted when given one spare hidden dimension. Our work turns one property of the input into a prediction of whether a network learns memory and explains why correlated data turns recurrent networks into change detectors. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.00420 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T06:41:55.779Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).