Metraipolis › archive › dc_3df9358bd765

Finite Spectral Depth Controls Gaussian Entrance Rank and Avoidance of the First Gradient-Flow Saddle — arXiv:2611.05022

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.05022
License (as archived)
CC0-1.0
Archived
2026-11-04T21:51:52.219Z
SHA-256 of the archived text
1e301400ffb76b424768acf85730d98e6c6273c141c4920957d528c21c77d826
Stable URL of this page
https://metraipolis.com/documento/dc_3df9358bd765

How to cite

Finite Spectral Depth Controls Gaussian Entrance Rank and Avoidance of the First Gradient-Flow Saddle — arXiv:2611.05022 — arXiv (en), https://arxiv.org/abs/2611.05022 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3df9358bd765 (Metraipolis archive dc_3df9358bd765, sha256 1e301400ffb76b424768acf85730d98e6c6273c141c4920957d528c21c77d826)
BibTeX
@misc{dc3df9358bd765,
  title = {Finite Spectral Depth Controls Gaussian Entrance Rank and Avoidance of the First Gradient-Flow Saddle — arXiv:2611.05022},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05022},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3df9358bd765; sha256 1e301400ffb76b424768acf85730d98e6c6273c141c4920957d528c21c77d826},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05022. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Matrix factorization provides a simple model for studying the training dynamics of neural networks. Recent studies of gradient flow under small initialization have analyzed how trajectories escape from a neighborhood of the origin and approach their first nonzero stationary point. In this paper, we study continuous-time dynamics that applies a finite number of Newton-Schulz iterations to the normalized matrix gradient (Muon). We show that the number of Newton-Schulz steps can change the rank selected during the early stages of training and allow the resulting trajectory to bypass the first nonzero stationary point approached by gradient flow. Under an additional residual-inertia condition, we prove that bypass occurs for all sufficiently large finite Newton-Schulz depths and provide an explicit sufficient threshold. We also construct an example in which the bypass threshold is four.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Optimization and Control (math.OC).
Pagina del lavoro: https://arxiv.org/abs/2611.05022 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T21:38:49.745Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).