Metraipolis › archive › dc_e27c489bf9be

Why Shallow Neural Networks Learn Sinusoidal Components First: An Orthogonal Polynomial Dynamics Framework — arXiv:2610.21964

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.21964
License (as archived): CC0-1.0
Archived: 2026-10-16T09:30:53.257Z
SHA-256 of the archived text: d7e53be2d304f9fa24e69b54233cdbdeb7bb02436b956fb6e34da675cdc077d4
Stable URL of this page: https://metraipolis.com/documento/dc_e27c489bf9be

How to cite

Why Shallow Neural Networks Learn Sinusoidal Components First: An Orthogonal Polynomial Dynamics Framework — arXiv:2610.21964 — arXiv (en), https://arxiv.org/abs/2610.21964 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e27c489bf9be (Metraipolis archive dc_e27c489bf9be, sha256 d7e53be2d304f9fa24e69b54233cdbdeb7bb02436b956fb6e34da675cdc077d4)

BibTeX

@misc{dce27c489bf9be,
  title = {Why Shallow Neural Networks Learn Sinusoidal Components First: An Orthogonal Polynomial Dynamics Framework — arXiv:2610.21964},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.21964},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e27c489bf9be; sha256 d7e53be2d304f9fa24e69b54233cdbdeb7bb02436b956fb6e34da675cdc077d4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21964. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Neural networks often learn low-frequency components before high-frequency ones --- a phenomenon called spectral bias or termed F-principle. Many numerical studies have demonstrated this phenomenon, and theoretical studies have sought to explain it. However, several goals remain insufficiently addressed: (1) explaining spectral bias asymptotically, including its origin and why the preferred components are generally sinusoidal; (2) extending the explanation to general activation functions, such as sigmoid and ReLU; (3) making weak assumptions about the data distribution; (4) avoiding aliasing effects, such as those arising in finite-domain Fourier analysis; and (5) developing a theoretical framework applicable to arbitrary networks. We address these goals theoretically by analyzing the eigenvalue decay and eigenvector shapes of the dynamics matrix H, which arises when we expand the network output as a general Fourier series (GFS) in an orthogonal polynomial basis and express the gradient-descent dynamics of the GFS coefficients as ordinary differential equations. The graded structure of H enables an intuitive explanation of spectral bias, while orthogonal polynomial theory helps us analyze how learning bias depends on the activation function and, potentially, the network architecture.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Spectral Theory (math.SP); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2610.21964 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:44:18.014Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).