Metraipolis › archive › dc_6d237cb90cd0
Depth Enables Local Entropy: Quadratic Depth Dependence in Deep Variation-Norm ReLU Regression — arXiv:2608.17434
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.17434
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T14:28:39.620Z
- SHA-256 of the archived text
9c9793ac7008c25367e73aadbee7bab49dc0fb45cf245266a6acfbabbcbe7012- Stable URL of this page
https://metraipolis.com/documento/dc_6d237cb90cd0
How to cite
Depth Enables Local Entropy: Quadratic Depth Dependence in Deep Variation-Norm ReLU Regression — arXiv:2608.17434 — arXiv (en), https://arxiv.org/abs/2608.17434 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6d237cb90cd0 (Metraipolis archive dc_6d237cb90cd0, sha256 9c9793ac7008c25367e73aadbee7bab49dc0fb45cf245266a6acfbabbcbe7012)
BibTeX
@misc{dc6d237cb90cd0,
title = {Depth Enables Local Entropy: Quadratic Depth Dependence in Deep Variation-Norm ReLU Regression — arXiv:2608.17434},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.17434},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6d237cb90cd0; sha256 9c9793ac7008c25367e73aadbee7bab49dc0fb45cf245266a6acfbabbcbe7012},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17434. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We study Gaussian regression over the explicit vector-valued Parhi--Nowak deep-RBV^2 architecture with depth L, width w, layer-sum variation budget A, and output bound B. For this O(L w^2)-parameterized architecture, the known lower and upper bounds differ by one factor of depth. We construct a local packing showing that the quadratic depth dependence is intrinsic under an explicit sample-size-dependent radius condition. The packing has log-cardinality Omega(L^2 w^2 log w); its codewords lie in an O(lambda) L^2 ball and are pairwise Omega(lambda)-separated. The main ingredients are a bias-corrected bounded-coefficient approximation theorem and balanced amplification: multiplying a depth-D ReLU network by q can be implemented using one constant channel so that every coefficient grows by only q^(1/D). Translation to vector-valued RBV^2 blocks then has layer-sum cost O(D w^2 q^(1/D)). Gaussian Fano yields a radius-explicit lower bound governed by the output, testing, and representation scales. Under A=B=R, sigma proportional to R, and the stated radius condition, this gives minimax risk at least of order L^2 w^2 log(w) R^2/n. A pseudodimension-based finite-net upper bound gives O-tilde(L^2 w^2 R^2/n) for unbounded Gaussian responses. Thus the minimax risk has quadratic polynomial dependence on depth, up to logarithmic factors, and exhibits a transition to representation-limited behavior at smaller radius. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.17434 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T14:27:54.263Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).