Metraipolis › archive › dc_f92183cbce24

A Spectral Renormalization-Group View of Learning — arXiv:2611.00422

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.00422
License (as archived)
CC0-1.0
Archived
2026-11-03T10:45:54.268Z
SHA-256 of the archived text
8d83e105b23a317e5fed7c81681f587cf62d0f4b9b6b1784559332e6643aa570
Stable URL of this page
https://metraipolis.com/documento/dc_f92183cbce24

How to cite

A Spectral Renormalization-Group View of Learning — arXiv:2611.00422 — arXiv (en), https://arxiv.org/abs/2611.00422 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f92183cbce24 (Metraipolis archive dc_f92183cbce24, sha256 8d83e105b23a317e5fed7c81681f587cf62d0f4b9b6b1784559332e6643aa570)
BibTeX
@misc{dcf92183cbce24,
  title = {A Spectral Renormalization-Group View of Learning — arXiv:2611.00422},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00422},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f92183cbce24; sha256 8d83e105b23a317e5fed7c81681f587cf62d0f4b9b6b1784559332e6643aa570},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00422. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We study how the layers in deep neural networks (NNs) converge, using Wilsonian Renormalization-Group (RG) ideas to organize the spectral picture. During training, these layers often develop heavy-tailed power-law (PL) spectra, and the strongest checkpoints often appear near the PL exponent $\alpha\approx 2$. This pattern raises a simple question: why should this value mark a well-trained layer? The answer is both statistical and physical. Our statistical hypothesis is that $\alpha=2$ marks the boundary between stable if not optimal learning and overfitting via non-self-averaging. The physical answer is that the same value makes the spectral energy gain scale evenly across logarithmic spectral scales, which is the proposed Wilsonian marginal condition. We then apply the ERG trace-log condition, the proposed SETOL theory, which we justify by taking volume-preserving, scale-invariant transformations to remove arbitrary layer scale along the proposed RG flow. The resulting spectral RG flow connects the HTSR and SETOL theories to optimizers such as Muon, and motivates a new optimizer extension, WW-PGD. Together, these results point toward a spectral Renormalization-Group view of learning, in which NNs may approach shared spectral structure in the settings studied here.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Disordered Systems and Neural Networks (cond-mat.dis-nn); Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.00422 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:39:50.401Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).