Metraipolis › archive › dc_7f0872634486

Neural LoFi with Backward Coupling: A Spectral Theory of Cross-Layer Feature Learning — arXiv:2610.32849

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.32849
License (as archived)
CC0-1.0
Archived
2026-10-26T20:12:42.035Z
SHA-256 of the archived text
e3fd49f7861c24b62167c8dc7ecd80c07d66fc4ca04fd92c66ce29a226298c5d
Stable URL of this page
https://metraipolis.com/documento/dc_7f0872634486

How to cite

Neural LoFi with Backward Coupling: A Spectral Theory of Cross-Layer Feature Learning — arXiv:2610.32849 — arXiv (en), https://arxiv.org/abs/2610.32849 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7f0872634486 (Metraipolis archive dc_7f0872634486, sha256 e3fd49f7861c24b62167c8dc7ecd80c07d66fc4ca04fd92c66ce29a226298c5d)
BibTeX
@misc{dc7f0872634486,
  title = {Neural LoFi with Backward Coupling: A Spectral Theory of Cross-Layer Feature Learning — arXiv:2610.32849},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.32849},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7f0872634486; sha256 e3fd49f7861c24b62167c8dc7ecd80c07d66fc4ca04fd92c66ce29a226298c5d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32849. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

During joint training, backpropagation allows later layers to reshape earlier representations, a feedback mechanism absent from purely layer-wise learning. We extend Neural Low-Degree Filtering (Neural LoFi), a recently proposed layer-wise spectral surrogate for feature learning, to capture this interaction. Starting from the exact backpropagation gradients, we derive a local coupling between a neuron's incoming weights and its corresponding next-layer column. In a controlled small-initialization regime with smooth activations and fixed surroundings, these dynamics amplify singular modes of a cross-layer correlation operator. Each growing mode pairs an earlier feature with an upper-layer response through shared neuron coefficients. This motivates a backward-coupling algorithm in which upper-layer sensitivities generate sample-dependent effective labels for earlier spectral steps, while corrected features refine upward transmission. A function-space variational principle characterizes the selected feature-response pairs by maximizing the label correlation of their product under kernel-norm constraints. For three-layer networks, we derive self-consistent continuous-time dynamics for the filtered hidden-layer features, the activation kernels, and the network output under the proposed backward coupling. In a hierarchical multi-index model, we establish a feature-recovery separation: backward coupling recovers lower-layer directions invisible to the original forward label-weighted statistic. The linear specialization recovers residual-dependent singular-mode dynamics, while paired modes provide a dynamical mechanism for linked low-rank covariance components consistent with the rainbow-network picture. Experiments with fully connected and convolutional networks on binary CIFAR-10 show improved accuracy over forward Neural LoFi at matched widths.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2610.32849 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-26T19:49:03.433Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).