Metraipolis › archive › dc_be046e753ed3
Muon with Finite Newton-Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization — arXiv:2608.26288
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.26288
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T16:09:14.206Z
- SHA-256 of the archived text
5b1db8919e168ecd1932b4e95a100825cf4daa325bfa6a29cebda823d46421e9- Stable URL of this page
https://metraipolis.com/documento/dc_be046e753ed3
How to cite
Muon with Finite Newton-Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization — arXiv:2608.26288 — arXiv (en), https://arxiv.org/abs/2608.26288 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_be046e753ed3 (Metraipolis archive dc_be046e753ed3, sha256 5b1db8919e168ecd1932b4e95a100825cf4daa325bfa6a29cebda823d46421e9)
BibTeX
@misc{dcbe046e753ed3,
title = {Muon with Finite Newton-Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization — arXiv:2608.26288},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.26288},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_be046e753ed3; sha256 5b1db8919e168ecd1932b4e95a100825cf4daa325bfa6a29cebda823d46421e9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.26288. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Muon has emerged as a strong optimizer for the matrix-valued parameters in large language model pretraining, approximately orthogonalizing its momentum with a few Newton-Schulz iterations. Existing theory either replaces this iteration with the exact polar factor it approximates, or treats its finite depth as an approximation error, and thus the iteration Muon actually runs can only hurt the guarantees. We show that finite Newton-Schulz can instead be beneficial for nonsmooth nonconvex optimization. To this end, we analyze Muon through the online-to-nonconvex conversion, which views the update rule as an online learner and converts its regret bound into a stationarity guarantee. The finite Newton-Schulz iteration smooths the discontinuous polar map into a Lipschitz map of the singular values, and Muon with finite Newton-Schulz can be regarded as an online learner with a smoothed spectral potential. This smoothing is exactly what the conversion needs: we prove that a Newton-Schulz depth growing only logarithmically in the target accuracy suffices for convergence to stationary points in nonsmooth nonconvex optimization, whereas Muon with the exact-polar update may fail to converge. The resulting sample complexity bounds match the best-known guarantees for nonsmooth nonconvex optimization and are optimal for smooth nonconvex optimization up to problem-dependent factors. The argument extends beyond Newton-Schulz to general spectral maps with the same smoothing property. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.LG). Categorie arXiv: Machine Learning (cs.LG) ; Optimization and Control (math.OC); Machine Learning (stat.ML). Pagina del lavoro: https://arxiv.org/abs/2608.26288 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T15:34:49.887Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).