Metraipolis › archive › dc_918daed16df9

Muon Meets Mamba: Spectral Optimization for State Space Models — arXiv:2608.03941

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.03941
License (as archived)
CC0-1.0
Archived
2026-10-31T12:42:36.020Z
SHA-256 of the archived text
f24a14ff31630036bef7fcd167a6a56a1e421af629e2e9f3cdec2ee87160605d
Stable URL of this page
https://metraipolis.com/documento/dc_918daed16df9

How to cite

Muon Meets Mamba: Spectral Optimization for State Space Models — arXiv:2608.03941 — arXiv (en), https://arxiv.org/abs/2608.03941 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_918daed16df9 (Metraipolis archive dc_918daed16df9, sha256 f24a14ff31630036bef7fcd167a6a56a1e421af629e2e9f3cdec2ee87160605d)
BibTeX
@misc{dc918daed16df9,
  title = {Muon Meets Mamba: Spectral Optimization for State Space Models — arXiv:2608.03941},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.03941},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_918daed16df9; sha256 f24a14ff31630036bef7fcd167a6a56a1e421af629e2e9f3cdec2ee87160605d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.03941. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Muon is a recent optimizer that orthogonalizes the update to each weight matrix with a Newton-Schulz iteration, which performs steepest descent under the spectral norm. Almost all the evidence for it comes from Transformer models, and its behavior on state-space models is largely unreported. We compare Muon with AdamW on Mamba-2 130M under a controlled protocol that varies only which weight groups are trained with Muon. The benefit is localized. Muon on the output projection alone beats Muon on the input projection or on both. The advantage is mainly one of token efficiency. It holds on two corpora and two token budgets, and persists when training continues well past the compute-optimal point. Conditioning does not explain the gain. Muon lowers the condition number of whichever projection it trains, but the better-conditioned input projection is not the one that helps.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.03941 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T12:31:30.384Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).