Metraipolis › archive › dc_73fe1cbcf6bd

Prospective Coding Improves Learning in Deep Continuous-Time Recurrent Networks — arXiv:2609.04134

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.04134
License (as archived)
CC0-1.0
Archived
2026-10-17T02:08:31.429Z
SHA-256 of the archived text
3d7d1bcf8323735dc4b43a928c5db3a498808f42a5fe384277ee64cc4525c6fd
Stable URL of this page
https://metraipolis.com/documento/dc_73fe1cbcf6bd

How to cite

Prospective Coding Improves Learning in Deep Continuous-Time Recurrent Networks — arXiv:2609.04134 — arXiv (en), https://arxiv.org/abs/2609.04134 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_73fe1cbcf6bd (Metraipolis archive dc_73fe1cbcf6bd, sha256 3d7d1bcf8323735dc4b43a928c5db3a498808f42a5fe384277ee64cc4525c6fd)
BibTeX
@misc{dc73fe1cbcf6bd,
  title = {Prospective Coding Improves Learning in Deep Continuous-Time Recurrent Networks — arXiv:2609.04134},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.04134},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_73fe1cbcf6bd; sha256 3d7d1bcf8323735dc4b43a928c5db3a498808f42a5fe384277ee64cc4525c6fd},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04134. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Temporal integration gives continuous-time recurrent networks memory, but in deep stacks it also delays bottom-up signals and attenuates top-down errors. We develop Recursive Quadrature Filters (RQFs), biologically motivated complex-valued temporal filters that are a special case of diagonal state-space models (SSMs), and ask whether this failure mode can be addressed by making each layer's bottom-up input prospective. Starting from an energy model, we derive the RQF dynamics and show that each RQF is a band-pass filter whose learnable parameters control its tuning frequency and bandwidth. We then make each layer's bottom-up input prospective using a parameter-free two-tap update that leaves the recurrent transition and parallel scan unchanged. We extend this correction to general diagonal SSMs and show that it mitigates depth-dependent gradient attenuation when temporal gradients are truncated, i.e., spatial-only backpropagation. We evaluate the intervention in RQFs, S5, and ORGaNICs (a nonlinear gated RNN) trained using full backpropagation through time (BPTT) and spatial-only backpropagation. Under full BPTT, prospective variants match or outperform their non-prospective controls in every model and configuration. A non-residual width-32 six-layer RQF reaches 96.09% accuracy on raw-audio Speech Commands with 31.9k parameters; a width-64 six-layer RQF reaches 83.56% on the 16,384-step Path-X task. These results identify RQFs as a parameter-efficient recurrent substrate and prospective-input coding as an input-side correction for deep continuous-time recurrent networks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-04 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Neural and Evolutionary Computing (cs.NE); Neurons and Cognition (q-bio.NC).
Pagina del lavoro: https://arxiv.org/abs/2609.04134 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T01:47:25.586Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-04?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).