Metraipolis › archive › dc_8e7eeebd5fc6

Ideal Paths for Approximating Logistic Gradient Descent Trajectories at Large Initialization — arXiv:2610.04142

Source
arXiv
Original
https://arxiv.org/abs/2610.04142
License (as archived)
CC0-1.0
Archived
2026-10-06T05:30:26.549Z
SHA-256 of the archived text
0523ed69fba743ffb87c1aea287b3918a2ace18171834583ec9cdaa7e442bc73
Stable URL of this page
https://metraipolis.com/documento/dc_8e7eeebd5fc6

How to cite

Ideal Paths for Approximating Logistic Gradient Descent Trajectories at Large Initialization — arXiv:2610.04142 — arXiv, https://arxiv.org/abs/2610.04142 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8e7eeebd5fc6 (Metraipolis archive dc_8e7eeebd5fc6, sha256 0523ed69fba743ffb87c1aea287b3918a2ace18171834583ec9cdaa7e442bc73)
BibTeX
@misc{dc8e7eeebd5fc6,
  title = {Ideal Paths for Approximating Logistic Gradient Descent Trajectories at Large Initialization — arXiv:2610.04142},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.04142},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8e7eeebd5fc6; sha256 0523ed69fba743ffb87c1aea287b3918a2ace18171834583ec9cdaa7e442bc73},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04142. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Modern training on a new task often starts from a previously trained model rather than from scratch, raising the question of how this initialization affects the subsequent training trajectory. Classical implicit-bias results characterize the direction selected by prolonged training, but this direction alone does not provide information regarding the intermediate behavior. We address this question through a geometric approximation of full-batch logistic gradient descent (GD) trajectories on strictly linearly separable data, with large initialization of scale $R$ motivated by prior training. From any limiting normalized initial position, we use minimum-norm projection rules to construct a unique continuous ideal path consisting of finitely many linear segments. The path has two stages: negative-margin correction followed by minimum-margin growth. We prove that, after an explicit two-stage time reparameterization, the fixed-step GD trajectory divided by $R$ converges uniformly to this path on every fixed parameter interval as $R\to\infty$. Further, our quantitative error bounds account for initialization perturbations and the transition between stages. This approximation provides asymptotic formulas for peak evaluation loss and cumulative training loss. In particular, peak evaluation loss can grow linearly in $R$ even when both endpoint losses tend to zero. The cumulative losses in the correction and margin-growth stages, normalized by $R^2$ and $R$, respectively, converge to explicit limits. Experiments on controlled geometries and fixed image features complement our theoretical results.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Optimization and Control (math.OC).
Pagina del lavoro: https://arxiv.org/abs/2610.04142 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:27:05.659Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).