Metraipolis › archive › dc_bcf44516af55
Dimension-Corrected Hitting Times for Heavy-Tailed Spectral Emergence in Neural Optimizer Dynamics — arXiv:2609.12994
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.12994
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T03:15:53.427Z
- SHA-256 of the archived text
a114655a2d98c75856b038abff4de236edfa682ad7378d5c0d3a9edf11542f35- Stable URL of this page
https://metraipolis.com/documento/dc_bcf44516af55
How to cite
Dimension-Corrected Hitting Times for Heavy-Tailed Spectral Emergence in Neural Optimizer Dynamics — arXiv:2609.12994 — arXiv (en), https://arxiv.org/abs/2609.12994 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bcf44516af55 (Metraipolis archive dc_bcf44516af55, sha256 a114655a2d98c75856b038abff4de236edfa682ad7378d5c0d3a9edf11542f35)
BibTeX
@misc{dcbcf44516af55,
title = {Dimension-Corrected Hitting Times for Heavy-Tailed Spectral Emergence in Neural Optimizer Dynamics — arXiv:2609.12994},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.12994},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bcf44516af55; sha256 a114655a2d98c75856b038abff4de236edfa682ad7378d5c0d3a9edf11542f35},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12994. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Heavy-tailed empirical spectral densities of neural-network weight matrices are widely used as diagnostics of implicit self-regularization, but the step complexity of heavy-tail emergence remains poorly understood. We formulate spectral heavy-tail formation as a right-censored hitting-time problem: a run that does not reach a heavy-tail diagnostic within the observation horizon is treated as censored rather than discarded. In controlled full-batch teacher--student dynamics, we find that the first-step spike--bulk gap alone does not explain onset time. Instead, finite-onset regression supports a dimension-corrected spectral-gap law, (\tau_{\mathrm{HT}}\approx C\Delta_1^{-\gamma}d^\rho), with (R^2=0.683), (\gamma=0.626), and (\rho=0.772) across 330 completed runs. Right-censored lognormal accelerated-failure-time models further favor the dimension-corrected model over a gap-only model, improving AIC from 706.62 to 628.70. Theoretically, we prove that exact early loss dynamics in linear networks do not determine factor spectral tails, that Adam recurrences alone do not imply spectral redistribution, and that projected singular-basis spreading implies contraction of a spectral-tail potential and hence a dimension-corrected hitting-time bound. Empirically, projected-kernel profiles support the sufficient spreading mechanism, Adam and AdamW agree under tested grids, GD and signGD do not reach onset in the same regimes, and real pretrained Qwen2.5-0.5B and Pythia-70M transformer weights show non-Gaussian spectral-tail structure relative to matched Gaussian nulls. The result is a reproducible spectral hitting-time law with rigorous conditional theory, not a claim that Adam necessarily generates heavy tails from first principles.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.12994 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T03:05:44.076Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).