Metraipolis › archive › dc_310f443569ea

SoftServe: A Scalable Quasi-Newton Method for Deep Learning — arXiv:2610.02182

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.02182
License (as archived)
CC0-1.0
Archived
2026-10-06T12:16:22.878Z
SHA-256 of the archived text
ded48f813eae5874e422dc56410635f2b1bde3aec960051980fb9185fc8ae691
Stable URL of this page
https://metraipolis.com/documento/dc_310f443569ea

How to cite

SoftServe: A Scalable Quasi-Newton Method for Deep Learning — arXiv:2610.02182 — arXiv (en), https://arxiv.org/abs/2610.02182 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_310f443569ea (Metraipolis archive dc_310f443569ea, sha256 ded48f813eae5874e422dc56410635f2b1bde3aec960051980fb9185fc8ae691)
BibTeX
@misc{dc310f443569ea,
  title = {SoftServe: A Scalable Quasi-Newton Method for Deep Learning — arXiv:2610.02182},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.02182},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_310f443569ea; sha256 ded48f813eae5874e422dc56410635f2b1bde3aec960051980fb9185fc8ae691},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.02182. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Quasi-Newton (QN) methods have long been among the most effective methods for large-scale unconstrained convex optimization. Two obstacles have limited their use in deep learning: non-convexity and enormous parameter sizes. We introduce SoftServe, a family of QN methods designed to overcome these obstacles without line searches or ad hoc curvature corrections. SoftServe derives positivedefinite curvature estimates from the variational objective of Berglund et al. (2025), even in the presence of negative curvature. We develop diagonal and Kroneckerfactored variants that preserve positive definiteness by construction and scale to massive neural networks. Finally, SoftServe relies on the stable coupled Newton-Schulz iteration for the required matrix operations, replacing costly matrix decompositions with GPU-friendly matrix multiplications. SoftServe excels on problems that are severely ill-conditioned, including tasks such as recurrent networks, deep autoencoders, physics-informed neural networks, and a 136M-parameter physics-informed diffusion model, often achieving lower losses than established baselines including Adam, Muon, and SOAP.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.02182 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:28.254Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).