Metraipolis › archive › dc_4b7faeb25b54
Signal-Noise Factorization Isolates Nuisance Variation into Removable Subspaces — arXiv:2610.00751
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.00751
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T15:33:35.349Z
- SHA-256 of the archived text
d5a31a3497d7a687c49174d5dec08335e62065ad6ac707e709fdb292c5f978d1- Stable URL of this page
https://metraipolis.com/documento/dc_4b7faeb25b54
How to cite
Signal-Noise Factorization Isolates Nuisance Variation into Removable Subspaces — arXiv:2610.00751 — arXiv (en), https://arxiv.org/abs/2610.00751 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4b7faeb25b54 (Metraipolis archive dc_4b7faeb25b54, sha256 d5a31a3497d7a687c49174d5dec08335e62065ad6ac707e709fdb292c5f978d1)
BibTeX
@misc{dc4b7faeb25b54,
title = {Signal-Noise Factorization Isolates Nuisance Variation into Removable Subspaces — arXiv:2610.00751},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.00751},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4b7faeb25b54; sha256 d5a31a3497d7a687c49174d5dec08335e62065ad6ac707e709fdb292c5f978d1},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.00751. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Recent theoretical work identified fundamental properties of representation geometry that shape inference ability of deep neural networks. These include signal-noise factorization (SNF), the ability to segregate signal from noise, and signal-signal factorization (SSF), the ability to segregate task-specific and task-irrelevant signals. Here, we built regularizers that reinforce these two properties during training. We compared networks trained with these regularizers to $L_2$-regularized baseline networks on the CIFAR-100 classification task to understand how our regularizers shape representation geometry and impact performance on a well-known computer vision baseline. Enhancing SNF via regularization improved model performance but enhancing SSF did not. Motivated by biomedical applications, we investigated how our regularizers affected performance on the BloodMNIST dataset treated with MedMNIST-C corruptions at five severity levels, and found even larger performance gains using the SNF regularizer. To understand the mechanism by which SNF-regularization produces improved performance, we analyzed the nuisance subspaces across regularization regimes, finding that the SNF-regularized models represent noise in distinct subspaces, separate from class-relevant signal. Because this geometry is explicit, the dominant corruption-induced directions can be estimated on held-out data and projected out of the representations. This manipulation led to a substantial gain in accuracy. These results show that regularizers that enforce signal-noise factorization can produce substantial improvements on computer vision tasks that contain out-of-distribution image distortions at inference time. They also highlight how shaping representations affects model performance: isolating nuisance variables from categorical ones is more important than maintaining factorized representations of categorical variables. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-10-02 in stat.ML). Categorie arXiv: Machine Learning (cs.LG) ; Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML). Pagina del lavoro: https://arxiv.org/abs/2610.00751 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T15:12:55.893Z dalla pagina https://arxiv.org/catchup/stat.ML/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).