Metraipolis › archive › dc_4fd7c47cfd79
Informative Fisher Modes in Neural Networks as a Result of Explicit Symmetry Breaking — arXiv:2611.00745
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00745
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:49:25.660Z
- SHA-256 of the archived text
ef46fd8808c4847942ab397f4d13064a240b6b952f6e7f292c8c7d864d672d42- Stable URL of this page
https://metraipolis.com/documento/dc_4fd7c47cfd79
How to cite
Informative Fisher Modes in Neural Networks as a Result of Explicit Symmetry Breaking — arXiv:2611.00745 — arXiv (en), https://arxiv.org/abs/2611.00745 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4fd7c47cfd79 (Metraipolis archive dc_4fd7c47cfd79, sha256 ef46fd8808c4847942ab397f4d13064a240b6b952f6e7f292c8c7d864d672d42)
BibTeX
@misc{dc4fd7c47cfd79,
title = {Informative Fisher Modes in Neural Networks as a Result of Explicit Symmetry Breaking — arXiv:2611.00745},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00745},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4fd7c47cfd79; sha256 ef46fd8808c4847942ab397f4d13064a240b6b952f6e7f292c8c7d864d672d42},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00745. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Learning in a neural network takes place on a high dimensional, non-convex loss landscape, the Hessian curvature of which dictates the learning dynamics. Despite its high dimensionality, the large number of architectural symmetries explains some of the geometric of this landscape, in particular flat directions, i.e. near-zero modes of the Hessian. These architectural symmetries have therefore been suggested as an organizing principle of deep learning. It remains an open question, though, wether there are additional organizing principles in the remaining degrees of freedom in the learning dynamic. Here we identify task-dependent contingent symmetries as such an additional organizing principle. We show that non-flat learning directions emerge due to explicit breaking of these contingent symmetries. We isolate this symmetry in an analytically solvable minimal model. Furthermore, we show that, by tuning an L2 penalty, signatures of the symmetry breaking become measurable. As a consequence of the same mechanism, adding noise to the data drives the model through a sequence of broken symmetry recoveries in the form of Baik-Ben-Arous-Péché (BBP) transitions. We demonstrate analytically for linear and numerically for deep linear networks that the contingent symmetry group remains explicit, and that the symmetry-breaking mechanism identifies which Hessian modes carry task-specific information. These signatures are measurable without a symmetry-group description and, hence, offer a route to study symmetry breaking as a more general principle in nonlinear networks and other learning systems. Tema degli esploratori: fisica e astronomia (elenco arXiv cond-mat.stat-mech, nuovi lavori del giorno). Categorie arXiv: Disordered Systems and Neural Networks (cond-mat.dis-nn) ; Statistical Mechanics (cond-mat.stat-mech); Mathematical Physics (math-ph). Pagina del lavoro: https://arxiv.org/abs/2611.00745 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T11:09:52.319Z dalla pagina https://arxiv.org/list/cond-mat.stat-mech/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).