Metraipolis › archive › dc_ecb96791c154

Physical Muon: Orthogonalization as an Equilibrium Computation — arXiv:2609.37525

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.37525
License (as archived): CC0-1.0
Archived: 2026-10-07T17:31:32.568Z
SHA-256 of the archived text: 0c6f76a7c1c49538ab0da0883defc8c69564ea2408b5a13c4d5f0792f0d10632
Stable URL of this page: https://metraipolis.com/documento/dc_ecb96791c154

How to cite

Physical Muon: Orthogonalization as an Equilibrium Computation — arXiv:2609.37525 — arXiv (en), https://arxiv.org/abs/2609.37525 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ecb96791c154 (Metraipolis archive dc_ecb96791c154, sha256 0c6f76a7c1c49538ab0da0883defc8c69564ea2408b5a13c4d5f0792f0d10632)

BibTeX

@misc{dcecb96791c154,
  title = {Physical Muon: Orthogonalization as an Equilibrium Computation — arXiv:2609.37525},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37525},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ecb96791c154; sha256 0c6f76a7c1c49538ab0da0883defc8c69564ea2408b5a13c4d5f0792f0d10632},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37525. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Physical neural networks and analog in-memory computing could reduce the energy cost of neural network training. Realizing this potential, however, requires optimizers that combine effective learning with physical implementability. SGD fits local analog updates but struggles on transformers, while Adam family is unstable against analog bias. Muon offers strong training performance, but its Newton--Schulz orthogonalization relies on dense matrix-matrix products. To address this obstacle, we introduce Physical Muon, which computes the orthogonalization as the equilibrium of a continuous-time flow. Random probes approximate the flow using matrix-vector products, reciprocal reads, and local rank-1 writes. To test whether this replacement preserves training performance, we evaluate it on a 10.95M-parameter transformer. The dense flow's mean validation cross-entropy is 0.0085 above Newton--Schulz across nine seeds per method; the probe implementation is 0.0188 above the control across two seeds. Circuit simulations further reproduce the flow dynamics and yield comparable training behavior.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Emerging Technologies (cs.ET); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.37525 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T16:59:48.341Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).