Metraipolis › archive › dc_261bdcc3bb06

Structural Limits of the Information-Theoretic Uncertainty Decomposition — arXiv:2609.39591

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.39591
License (as archived): CC0-1.0
Archived: 2026-10-07T14:46:15.407Z
SHA-256 of the archived text: b1045210a01565384a9e361045f785758c3a6cc92d30608796596f2b1efac146
Stable URL of this page: https://metraipolis.com/documento/dc_261bdcc3bb06

How to cite

Structural Limits of the Information-Theoretic Uncertainty Decomposition — arXiv:2609.39591 — arXiv (en), https://arxiv.org/abs/2609.39591 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_261bdcc3bb06 (Metraipolis archive dc_261bdcc3bb06, sha256 b1045210a01565384a9e361045f785758c3a6cc92d30608796596f2b1efac146)

BibTeX

@misc{dc261bdcc3bb06,
  title = {Structural Limits of the Information-Theoretic Uncertainty Decomposition — arXiv:2609.39591},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.39591},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_261bdcc3bb06; sha256 b1045210a01565384a9e361045f785758c3a6cc92d30608796596f2b1efac146},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39591. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Uncertainty estimation in machine learning typically decomposes uncertainty into aleatoric uncertainty (AU) and epistemic uncertainty (EU) using the standard information-theoretic framework. However, in practice, two critical issues arise: entanglement (AU and EU are highly correlated) and epistemic collapse (EU magnitude shrinks with increasing model capacity). We analyze this framework on a functional level and discover that significant portions of the assumed AU, EU range are infeasible in finite settings, and cannot be attained with any class probabilities. We characterize how this infeasible region scales with the number of classes and Monte Carlo samples $N$ (e.g., from ensembles with $N$ members), revealing it is bounded by $\text{AU} \leq \log(2)/N$. Crucially, the infeasible region's boundary helps explain epistemic collapse: when model confidence is high, $\text{AU} > \text{EU}$ is guaranteed by this fundamental structural limitation. Our findings show that increasing ensemble size mitigates epistemic collapse by reducing the infeasible area. Lastly, we caution against interpreting AU and EU as independent quantities in low AU regimes, since we show they are coupled when $\text{AU} \leq \log(2)/N$.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-10-01 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.39591 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T14:19:00.772Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).