Metraipolis › archive › dc_ab0ece2c351b

Double descent is the principle of least action — arXiv:2609.19076

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.19076
License (as archived)
CC0-1.0
Archived
2026-10-15T15:05:01.067Z
SHA-256 of the archived text
6c9b07a4ef37bcd34a8144a0c8737ce5a4b67864fb5424d01dc6d99d6c2f6077
Stable URL of this page
https://metraipolis.com/documento/dc_ab0ece2c351b

How to cite

Double descent is the principle of least action — arXiv:2609.19076 — arXiv (en), https://arxiv.org/abs/2609.19076 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ab0ece2c351b (Metraipolis archive dc_ab0ece2c351b, sha256 6c9b07a4ef37bcd34a8144a0c8737ce5a4b67864fb5424d01dc6d99d6c2f6077)
BibTeX
@misc{dcab0ece2c351b,
  title = {Double descent is the principle of least action — arXiv:2609.19076},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.19076},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ab0ece2c351b; sha256 6c9b07a4ef37bcd34a8144a0c8737ce5a4b67864fb5424d01dc6d99d6c2f6077},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19076. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The test error of a model plotted against its number of parameters $d$ falls, peaks when the model can just fit the training data, and falls again, exhibiting the double descent phenomenon. We explain the phenomenon with statistical mechanics. The training trajectory of a stochastic gradient-based method is a particle wandering over the energy landscape of the training loss at an induced temperature $T$, and a run that has equilibrated visits every parameter vector of a given training loss equally often, the fundamental postulate of statistical mechanics, with probability given by the Boltzmann distribution. Because training starts at an initial point and has only finite time to diffuse, it carries an effective weight decay, which makes every parameter a quadratic degree of freedom. The equipartition theorem then distributes the energy among the $d$ degrees of freedom in shares of $T/2$, so at a fixed training loss adding parameters lowers the temperature and drives the Boltzmann distribution toward the stationary path. Finally, adding parameters can only lower the $L^2$ norm of the stationary path, so a solution sampled at fixed loss is less likely to be large with increasing $d$, effectively increasing weight regularization.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Statistics Theory (math.ST); Computational Physics (physics.comp-ph); Data Analysis, Statistics and Probability (physics.data-an).
Pagina del lavoro: https://arxiv.org/abs/2609.19076 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T14:08:19.175Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).