Metraipolis › archive › dc_466962b58014
Explaining f-Divergence-Based Regularization via Local Curvature and Sharpness-Aware Minimization — arXiv:2609.09367
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.09367
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T10:53:26.300Z
- SHA-256 of the archived text
7be699060fe952ebda780b1b30f59f99a7a294597638e87e8708de9126033ce2- Stable URL of this page
https://metraipolis.com/documento/dc_466962b58014
How to cite
Explaining f-Divergence-Based Regularization via Local Curvature and Sharpness-Aware Minimization — arXiv:2609.09367 — arXiv (en), https://arxiv.org/abs/2609.09367 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_466962b58014 (Metraipolis archive dc_466962b58014, sha256 7be699060fe952ebda780b1b30f59f99a7a294597638e87e8708de9126033ce2)
BibTeX
@misc{dc466962b58014,
title = {Explaining f-Divergence-Based Regularization via Local Curvature and Sharpness-Aware Minimization — arXiv:2609.09367},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.09367},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_466962b58014; sha256 7be699060fe952ebda780b1b30f59f99a7a294597638e87e8708de9126033ce2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09367. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Divergence-based regularization and Sharpness-Aware Minimization (SAM) are two prominent approaches for improving generalization in deep learning, both motivated by robustness to perturbations. However, their relationship has remained largely unexplored. Building on classical second-order expansions of $f$-divergences, we show that the two methods are locally consistent under parameter-space perturbations: both induce curvature-sensitive penalties, with divergence regularization yielding a Fisher-weighted quadratic form and SAM penalizing sharpness through the dominant Hessian eigenvalue. For negative log-likelihood objectives with exponential-family output distributions, this correspondence becomes especially transparent, since the Fisher and Gauss-Newton matrices coincide. We further show that the same local geometric perspective extends to input-space perturbations, where divergence-based regularization is defined through transformations of the input. In this setting, the regularizer induces a pullback quadratic form on the input space, providing a more general perturbation framework than standard SAM while preserving the same local sensitivity interpretation. To validate the analysis empirically, we use the asymmetric $\alpha$-skew Jensen-Shannon divergence (JSD) family as a controlled testbed. Its local curvature coefficient scales as $\alpha(1-\alpha)$ and is maximized at the symmetric point $\alpha=\tfrac12$, which recovers the standard JSD. Loss-landscape visualizations in the input-perturbation regime show that stronger induced curvature penalization is associated with flatter local minima. Experiments on four benchmark datasets further demonstrate that both accuracy and negative log-likelihood are consistently best near this regime of maximal curvature penalization. Tema degli esploratori: matematica e informatica teorica (retro-catalogo arXiv: annunci del 2026-09-10 in math.IT). Categorie arXiv: Machine Learning (cs.LG) ; Information Theory (cs.IT). Pagina del lavoro: https://arxiv.org/abs/2609.09367 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T09:00:25.852Z dalla pagina https://arxiv.org/catchup/math.IT/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).