Metraipolis › archive › dc_754c8b4bdfb4
Non-asymptotic implicit bias of logistic regression at early-stage gradient descent dynamics — arXiv:2608.04382
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.04382
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T11:34:47.884Z
- SHA-256 of the archived text
6b4062774ac89bba21fbc3b9a593f75224bcebde9050e96b83c4c568f141ce98- Stable URL of this page
https://metraipolis.com/documento/dc_754c8b4bdfb4
How to cite
Non-asymptotic implicit bias of logistic regression at early-stage gradient descent dynamics — arXiv:2608.04382 — arXiv (en), https://arxiv.org/abs/2608.04382 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_754c8b4bdfb4 (Metraipolis archive dc_754c8b4bdfb4, sha256 6b4062774ac89bba21fbc3b9a593f75224bcebde9050e96b83c4c568f141ce98)
BibTeX
@misc{dc754c8b4bdfb4,
title = {Non-asymptotic implicit bias of logistic regression at early-stage gradient descent dynamics — arXiv:2608.04382},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.04382},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_754c8b4bdfb4; sha256 6b4062774ac89bba21fbc3b9a593f75224bcebde9050e96b83c4c568f141ce98},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04382. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Gradient descent has been of particular interest in modern machine learning beyond sole focus on optimization. Implicit bias emerging from optimization, though not being encoded by the learning objective, often prevents from overfitting to spurious patterns. A typical instance is the max-margin implicit bias of a linear classifier, widely established for exponentially tailed loss functions. Even after having a given dataset separated, the parameter vector continues to evolve towards the max-margin direction asymptotically along the gradient descent dynamics. This phenomenon corroborates a frequent empirical observation of "train longer, generalize better." However, the max-margin convergence is an asymptotic phenomenon, and what is worse, this asymptotic convergence rate is significantly slower than pure convex optimization. Even so, the parameter vector along gradient descent dynamics commonly correlates with the max-margin direction positively (though not exactly) within considerably fewer iterations than the asymptotic rate. By shedding another light on this classical problem, this work aims to understand the mechanism of this early-stage alignment phenomenon. Our theoretical results demonstrate that the parameter vector weakly aligns with the max-margin direction within $O(\exp(\exp(-\delta)))$ iterations, where $\delta>0$ is the permissible alignment error, which is shown to be tight. By tracking the radial and tangential flows, our proof operates on the alignment dynamics directly with dataset geometry and gets rid of the asymptotic expansion, which is a key insight to establishing faster weak alignment. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-06 in cs.LG). Categorie arXiv: Machine Learning (cs.LG) ; Machine Learning (stat.ML). Pagina del lavoro: https://arxiv.org/abs/2608.04382 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T11:24:48.778Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).