Metraipolis › archive › dc_a0a895ea01cd

Constrained Nonconvex Stochastic Optimization with One Projection — arXiv:2609.34099

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.34099
License (as archived)
CC0-1.0
Archived
2026-10-07T23:06:40.649Z
SHA-256 of the archived text
ebdd326a124d8b30b38375b3c782c052e72bdefb6f4ede3b2253356aa047adb3
Stable URL of this page
https://metraipolis.com/documento/dc_a0a895ea01cd

How to cite

Constrained Nonconvex Stochastic Optimization with One Projection — arXiv:2609.34099 — arXiv (en), https://arxiv.org/abs/2609.34099 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a0a895ea01cd (Metraipolis archive dc_a0a895ea01cd, sha256 ebdd326a124d8b30b38375b3c782c052e72bdefb6f4ede3b2253356aa047adb3)
BibTeX
@misc{dca0a895ea01cd,
  title = {Constrained Nonconvex Stochastic Optimization with One Projection — arXiv:2609.34099},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.34099},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a0a895ea01cd; sha256 ebdd326a124d8b30b38375b3c782c052e72bdefb6f4ede3b2253356aa047adb3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34099. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Constrained nonconvex optimization has seen increasing application in modern machine learning, such as safe LLM alignment/finetuning, transfer learning and rank-constrained continual learning. The commonly used approach is Projected SGD which requires projection at every iteration. However, projection onto a functional constraint can cost substantially more than a stochastic first-order update. We study whether this operation can be deferred until the end of nonconvex stochastic optimization, and only do it once. For weakly convex, possibly nonsmooth objectives with regular convex constraints, we give a penalized proximal method that uses $\widetilde O(\epsilon^{-4})$ stochastic subgradients and one projection onto the constraint set. The returned point is exactly feasible and has a small Moreau-envelope stationarity measure; for smooth objectives, the same algorithm controls the projected-gradient mapping. For smooth nonconvex constraints, we assume a global lower bound on the infeasible constraint slope which permits arbitrary, possibly infeasible initialization. An exact-penalty variant attains the same stochastic-oracle order and returns an exactly feasible point within $O(\epsilon)$ of an $\epsilon$-KKT point. All intermediate updates use only projections onto a simple Euclidean ball. These are oracle-complexity guarantees: the cost of the single terminal projection is separate, and no efficient projection algorithm for a general nonconvex set is assumed to follow from regularity.

Tema degli esploratori: matematica e informatica teorica (retro-catalogo arXiv: annunci del 2026-09-29 in math.OC).
Categorie arXiv: Optimization and Control (math.OC).
Pagina del lavoro: https://arxiv.org/abs/2609.34099 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:42:39.239Z dalla pagina https://arxiv.org/catchup/math.OC/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).