Metraipolis › archive › dc_db21caafb8dc
Informational Frustration in Neural Manifolds: Shannon Bottlenecks and the Limits of Learnability — arXiv:2606.30512
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2606.30512
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T11:13:21.177Z
- SHA-256 of the archived text
53dcbcc023fff46931b6fd77eaece37d8c8403ad72a638d070f124e940558d3d- Stable URL of this page
https://metraipolis.com/documento/dc_db21caafb8dc
How to cite
Informational Frustration in Neural Manifolds: Shannon Bottlenecks and the Limits of Learnability — arXiv:2606.30512 — arXiv (en), https://arxiv.org/abs/2606.30512 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_db21caafb8dc (Metraipolis archive dc_db21caafb8dc, sha256 53dcbcc023fff46931b6fd77eaece37d8c8403ad72a638d070f124e940558d3d)
BibTeX
@misc{dcdb21caafb8dc,
title = {Informational Frustration in Neural Manifolds: Shannon Bottlenecks and the Limits of Learnability — arXiv:2606.30512},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2606.30512},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_db21caafb8dc; sha256 53dcbcc023fff46931b6fd77eaece37d8c8403ad72a638d070f124e940558d3d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2606.30512. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Why overparameterised deep networks generalise so remarkably well remains one of the most stubborn open questions in machine learning theory. Classical frameworks like VC dimension and Rademacher complexity predict catastrophic overfitting in modern models, leaving a massive theoretical gap between theory and reality. In this paper, we bridge this divide by introducing a unified framework that links information theory, topology, and statistical mechanics to map the hard limits of deep learning. Central to our approach is the Entropic Learnability Horizon (ELH): a fundamental law stating that a network can only truly learn a target function if the Shannon entropy of the data manifold outpaces the topological entropy of the function's decision boundary, balanced by the von Neumann entropy of the network's weight space. We establish the Shannon-Topological Bottleneck Theorem, proving that when a target boundary's geometric complexity exceeds this informational horizon, the system undergoes a sudden entropic phase transition. It falls into a state of Informational Frustration - a glassy, rigid memorization phase where generalization becomes thermodynamically impossible. Using this lens, we show that the enigmatic phenomenon of "grokking" is actually an Entropic Release, where weights abruptly reorganise to unlock the bottleneck. Finally, we translate this theory into practice with Entropic Gradient Descent (EGD), an optimization algorithm that dynamically manages weight entropy to keep learning on track. Ultimately, this work repositions entropy not just as a tool for tracking uncertainty but as the fundamental physical currency that dictates whether a machine can learn. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-06 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computational Complexity (cs.CC); Computational Geometry (cs.CG); Machine Learning (stat.ML). Pagina del lavoro: https://arxiv.org/abs/2606.30512 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T11:02:13.449Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).