Metraipolis › archive › dc_a93e8b06bc04
Grokking through the Lens of Minimum-Norm Interpolation — arXiv:2609.38453
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.38453
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T13:45:18.935Z
- SHA-256 of the archived text
9f05bd511699b937c87289197e5505b8c0aa6e821dc333115416e2de1bb970da- Stable URL of this page
https://metraipolis.com/documento/dc_a93e8b06bc04
How to cite
Grokking through the Lens of Minimum-Norm Interpolation — arXiv:2609.38453 — arXiv (en), https://arxiv.org/abs/2609.38453 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a93e8b06bc04 (Metraipolis archive dc_a93e8b06bc04, sha256 9f05bd511699b937c87289197e5505b8c0aa6e821dc333115416e2de1bb970da)
BibTeX
@misc{dca93e8b06bc04,
title = {Grokking through the Lens of Minimum-Norm Interpolation — arXiv:2609.38453},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.38453},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a93e8b06bc04; sha256 9f05bd511699b937c87289197e5505b8c0aa6e821dc333115416e2de1bb970da},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38453. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Grokking shows that fitting the training data and learning the underlying signal can occur at very different stages. However, existing theories offer limited quantitative insight into how this delayed generalization depends on inductive bias and signal structure. Our work addresses the gap by developing a statistical theory that characterizes how regularization geometry and signal sparsity govern generalization near interpolation. In particular, we focus on the prototypical setting of high-dimensional regression and identify regimes in which sparsity-promoting regularization makes exact interpolation much more accurate than approximate fitting. In strongly overparameterized noiseless problems, we prove a zero--one generalization law and construct a family of convex norms whose interpolators transition from the trivial risk of the all-zero predictor to exact recovery, while keeping the training error equal to $0$. Furthermore, when feature dimension and sample size are proportional, we provide a precise characterization of training and generalization errors along $\ell_r$-regularization paths. This in turn allows us to quantify the generalization gain that remains near interpolation: we show that this gain increases as the norm becomes more sparsity-promoting and as the target becomes sparser, with a sharp drop in generalization reached for noiseless data and $\ell_1$ regularization. Experiments on diagonal linear networks and transformers trained on modular arithmetic demonstrate the generality of our theoretical predictions. Finally, beyond grokking, our work reveals a statistical instability in minimum-norm interpolation: small perturbations in the regularization strength can lead to drastically different generalization, while preserving small training error. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.LG). Categorie arXiv: Machine Learning (cs.LG) ; Statistics Theory (math.ST); Machine Learning (stat.ML). Pagina del lavoro: https://arxiv.org/abs/2609.38453 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T13:18:10.212Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).