Metraipolis › archive › dc_46ccbbc6bfa1
A Function-Space Approach to the Statistical Mechanics of Learning Dynamics — arXiv:2609.09589
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.09589
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T07:24:26.134Z
- SHA-256 of the archived text
784d66cf4d5f0d0820b78aafd030e1851dd9e5faf5c1f60cd2080525c1ae2a89- Stable URL of this page
https://metraipolis.com/documento/dc_46ccbbc6bfa1
How to cite
A Function-Space Approach to the Statistical Mechanics of Learning Dynamics — arXiv:2609.09589 — arXiv (en), https://arxiv.org/abs/2609.09589 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_46ccbbc6bfa1 (Metraipolis archive dc_46ccbbc6bfa1, sha256 784d66cf4d5f0d0820b78aafd030e1851dd9e5faf5c1f60cd2080525c1ae2a89)
BibTeX
@misc{dc46ccbbc6bfa1,
title = {A Function-Space Approach to the Statistical Mechanics of Learning Dynamics — arXiv:2609.09589},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.09589},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_46ccbbc6bfa1; sha256 784d66cf4d5f0d0820b78aafd030e1851dd9e5faf5c1f60cd2080525c1ae2a89},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09589. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
In the kernel regime, neural-network learning inherits its preferences from a frozen spectrum. During feature learning, this spectrum evolves, yet networks retain systematic biases toward simple, smooth directions. We develop a function-space statistical framework explaining the origin of these preferences, treating functions and their learning operators as macroscopic variables, with parameterization entering through the multiplicity of parameter configurations realizing each function. For mean-squared loss, error relaxes exactly under the evolving learning operator $M=JJ^\ast$. Training stochasticity induces a Gaussian weight over function-space states, while parameter multiplicity contributes an entropic operator $B$, defined by the curvature of its log multiplicity. A local Laplace expansion yields the fluctuation free energy $\Phi_{\mathrm{fluc}}(M;B)=\frac{\sigma_\xi^2}{2}\log\det(M^{-1}+B)+\mathrm{const}$, analogous to an Occam factor. Under mild statistical conditions, this free energy is rotationally stationary exactly when $[M,B]=0$, is minimized by pairing large eigenvalues of $M$ with small eigenvalues of $B$, and generates a local restoring force against mismatch. Learning is therefore biased toward faster relaxation along entropically cheaper directions. This preference strengthens with training noise and vanishes in the deterministic limit, beyond gradient-flow accounts of operator alignment. For ReLU networks, we relate entropic curvature to the minimal rearrangement of activation boundaries required for a functional change and bound this structural cost by directional smoothness. Consequently, smooth directions are preferentially learned faster, in a data-adaptive manner, even as the learning operator evolves.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-10 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.09589 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T07:00:16.193Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).