Metraipolis › archive › dc_680f8893db7f

Percolation Dynamics in Optimization: Variance Cascades and Nested Symmetry — arXiv:2609.02373

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.02373
License (as archived)
CC0-1.0
Archived
2026-10-17T03:37:57.605Z
SHA-256 of the archived text
549111f5f84cb6b00f7fdd758c93f03b386b6af36279e5b1b016d5d01436ed02
Stable URL of this page
https://metraipolis.com/documento/dc_680f8893db7f

How to cite

Percolation Dynamics in Optimization: Variance Cascades and Nested Symmetry — arXiv:2609.02373 — arXiv (en), https://arxiv.org/abs/2609.02373 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_680f8893db7f (Metraipolis archive dc_680f8893db7f, sha256 549111f5f84cb6b00f7fdd758c93f03b386b6af36279e5b1b016d5d01436ed02)
BibTeX
@misc{dc680f8893db7f,
  title = {Percolation Dynamics in Optimization: Variance Cascades and Nested Symmetry — arXiv:2609.02373},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.02373},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_680f8893db7f; sha256 549111f5f84cb6b00f7fdd758c93f03b386b6af36279e5b1b016d5d01436ed02},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02373. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We study the dynamics of Stochastic Gradient Descent (SGD), which is known to steer deep neural networks toward invariant sets that correspond to simpler subnetworks. How this steering unfolds over time remains poorly understood. We answer this by modeling the stochastic gradient flow (SGF) as a percolation process, in which nested architectural symmetries force subnetworks to merge in discrete blocks rather than by single-edge attachment. These structural transitions register as variance spikes in a macroscopic order parameter echoing physical phase transitions. We further state sufficient conditions under which the trapping argument carries over to Adam and AdamW under heavy-tailed gradient noise and measure them on a trained Transformer.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech); Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.02373 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T03:13:12.543Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).