Metraipolis › archive › dc_e16bccb01338

A Spectral Renormalization-Group View of Learning — arXiv:2611.00422

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.00422
License (as archived)
CC0-1.0
Archived
2026-11-03T10:34:12.038Z
SHA-256 of the archived text
b14f510f547dc0fd80b89c4cdf3587d3ebc7fdf5ae4064d0fb1dc5fbb1da7e2e
Stable URL of this page
https://metraipolis.com/documento/dc_e16bccb01338

How to cite

A Spectral Renormalization-Group View of Learning — arXiv:2611.00422 — arXiv (en), https://arxiv.org/abs/2611.00422 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e16bccb01338 (Metraipolis archive dc_e16bccb01338, sha256 b14f510f547dc0fd80b89c4cdf3587d3ebc7fdf5ae4064d0fb1dc5fbb1da7e2e)
BibTeX
@misc{dce16bccb01338,
  title = {A Spectral Renormalization-Group View of Learning — arXiv:2611.00422},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00422},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e16bccb01338; sha256 b14f510f547dc0fd80b89c4cdf3587d3ebc7fdf5ae4064d0fb1dc5fbb1da7e2e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00422. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We study how the layers in deep neural networks (NNs) converge, using Wilsonian Renormalization-Group (RG) ideas to organize the spectral picture. During training, these layers often develop heavy-tailed power-law (PL) spectra, and the strongest checkpoints often appear near the PL exponent $\alpha\approx 2$. This pattern raises a simple question: why should this value mark a well-trained layer? The answer is both statistical and physical. Our statistical hypothesis is that $\alpha=2$ marks the boundary between stable if not optimal learning and overfitting via non-self-averaging. The physical answer is that the same value makes the spectral energy gain scale evenly across logarithmic spectral scales, which is the proposed Wilsonian marginal condition. We then apply the ERG trace-log condition, the proposed SETOL theory, which we justify by taking volume-preserving, scale-invariant transformations to remove arbitrary layer scale along the proposed RG flow. The resulting spectral RG flow connects the HTSR and SETOL theories to optimizers such as Muon, and motivates a new optimizer extension, WW-PGD. Together, these results point toward a spectral Renormalization-Group view of learning, in which NNs may approach shared spectral structure in the settings studied here.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Disordered Systems and Neural Networks (cond-mat.dis-nn); Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.00422 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:29:48.693Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).