Metraipolis › archive › dc_faf70babf4e9
Blockwise Stabilized Adaptive Cubic Regularization with Subsolvers via Recurrence — arXiv:2608.22129
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.22129
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T08:27:59.982Z
- SHA-256 of the archived text
97b8d157d08dd1f3efa84d3772e00dd62b62f05dbec89a1f1a4812e0afd60d01- Stable URL of this page
https://metraipolis.com/documento/dc_faf70babf4e9
How to cite
Blockwise Stabilized Adaptive Cubic Regularization with Subsolvers via Recurrence — arXiv:2608.22129 — arXiv (en), https://arxiv.org/abs/2608.22129 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_faf70babf4e9 (Metraipolis archive dc_faf70babf4e9, sha256 97b8d157d08dd1f3efa84d3772e00dd62b62f05dbec89a1f1a4812e0afd60d01)
BibTeX
@misc{dcfaf70babf4e9,
title = {Blockwise Stabilized Adaptive Cubic Regularization with Subsolvers via Recurrence — arXiv:2608.22129},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.22129},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_faf70babf4e9; sha256 97b8d157d08dd1f3efa84d3772e00dd62b62f05dbec89a1f1a4812e0afd60d01},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.22129. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Overparameterized networks often have ill-conditioned, saddle rich loss landscapes. They are usually trained with first-order methods that struggle on those landscapes, while many second-order methods stay infeasible at scale. Some scalable cubic methods replace the Hessian by a computationally cheap approximation and drop the exact block curvature. This paper introduces ARC-block, a blockwise adaptive regularization with cubics (ARC) optimizer. For each parameter tensor it minimizes a cubic model over the true block Hessian, with a per-block adaptive cubic constant and a monotone guard on the full loss. Large tensors are handled matrix-free in a Lanczos-built Krylov subspace. ARC-block's CubicKrylov subsolver's descent is monotone with a per-block cubic decrease bound. For exact steps, and for Krylov steps that meet the Cartis--Gould--Toint residual test, we provide an $\mathcal{O}(\sqrt{B}\varepsilon^{-3/2})$ sweep bound above a cross-block coupling floor. Another subsolver, ARC-$\varphi_1$, uses a trust-controlled exponential relaxation step. Among the cubic Newton methods evaluated here, these are the only ones whose steps stay exact on every parameter block (within the Krylov subspace), including on a $91.4$M-parameter implicit neural representation (INR). Run to convergence on FINER 2D image-fitting INR network, ARC-$\varphi_1$ reaches $133.5$ dB peak signal-to-noise ratio (PSNR), with no post peak collapse. Tuned Adam peaks at $78.2$ dB at $2{,}629$ s (step ${\sim}625$k, about $44$ minutes), further steps degrade its PSNR. On the same benchmark, SOAP, L-BFGS, and the scalable diagonal cubic method AdaCubic stay at or below Adam's $78.2$ dB peak. Over the benchmark's 20 images, run to a plateau rule, ARC-$\varphi_1$'s per-image PSNR has median $132.4$ and mean $131.5\pm3.3$ dB against tuned Adam's $72.8\pm4.5$ dB.
Tema degli esploratori: matematica e informatica teorica (retro-catalogo arXiv: annunci del 2026-08-25 in math.NA).
Categorie arXiv: Machine Learning (cs.LG) ; Numerical Analysis (math.NA).
Pagina del lavoro: https://arxiv.org/abs/2608.22129 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T08:27:23.861Z dalla pagina https://arxiv.org/catchup/math.NA/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).