Metraipolis › archive › dc_e02ff3d4b196
An Information Budget for Label-Noise Memorization, and What Statistical Complexity Tracks: A Study with Multilayer Perceptrons — arXiv:2611.04199
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.04199
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T12:40:47.045Z
- SHA-256 of the archived text
5e59cc372f7dfdfb9093272ebcda87140a121c245c0b39ee2f75403b8f3e350a- Stable URL of this page
https://metraipolis.com/documento/dc_e02ff3d4b196
How to cite
An Information Budget for Label-Noise Memorization, and What Statistical Complexity Tracks: A Study with Multilayer Perceptrons — arXiv:2611.04199 — arXiv (en), https://arxiv.org/abs/2611.04199 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e02ff3d4b196 (Metraipolis archive dc_e02ff3d4b196, sha256 5e59cc372f7dfdfb9093272ebcda87140a121c245c0b39ee2f75403b8f3e350a)
BibTeX
@misc{dce02ff3d4b196,
title = {An Information Budget for Label-Noise Memorization, and What Statistical Complexity Tracks: A Study with Multilayer Perceptrons — arXiv:2611.04199},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.04199},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e02ff3d4b196; sha256 5e59cc372f7dfdfb9093272ebcda87140a121c245c0b39ee2f75403b8f3e350a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04199. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
How much label noise has a network memorized, and when? We study a multilayer perceptron and what statistical complexity adds to a description of its predictions. Standard identities decompose predictive information into signal, memorization, and heterogeneity. The memorization term vanishes in the population under the stated noise-independence condition. A Fano-type bound relates it to training accuracies, but the memorized fraction M provides no upper bound. Across 89 MNIST runs with ten pre-specified hypotheses (five supported, four unsupported, one not testable), normalized memorization information rose with M on training samples, remained near zero on held-out samples with fresh noisy labels, and was close to its Fano bound. Unlike raw M, it has a zero population null under the independence condition and chance correction largely removed this difference. When every label was replaced by an incorrect class, test accuracy fell to 0.5%, yet predictions retained information about the true class. Mean statistical complexity peaked before memorization at corruption rates up to 40%. At higher rates, its peak fell within the memorization phase, near the epoch when mean training confidence reached 0.6 and the two could not be separated. As a stopping rule, it matched established rules only at low noise and detected corrupted samples less well than simple label-free scores. An exploratory extension of 280 runs covering Fashion-MNIST, pair-flip noise, and four training variants yielded similar results under uniform noise, but not pair-flip noise, where complexity peaked within two epochs. Conclusions are limited to this architecture, two small datasets, and synthetic noise. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.04199 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T12:29:47.285Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).