Metraipolis › archive › dc_52674aa48908

Sharp Approximation Rates for Neural Networks with Affine Latent Parameterizations — arXiv:2608.31157

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.31157
License (as archived)
CC0-1.0
Archived
2026-10-17T08:39:37.959Z
SHA-256 of the archived text
0ec535a25f2998ce53656f4e1d33582e7a09519babcffb6cff1c86035dbce112
Stable URL of this page
https://metraipolis.com/documento/dc_52674aa48908

How to cite

Sharp Approximation Rates for Neural Networks with Affine Latent Parameterizations — arXiv:2608.31157 — arXiv (en), https://arxiv.org/abs/2608.31157 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_52674aa48908 (Metraipolis archive dc_52674aa48908, sha256 0ec535a25f2998ce53656f4e1d33582e7a09519babcffb6cff1c86035dbce112)
BibTeX
@misc{dc52674aa48908,
  title = {Sharp Approximation Rates for Neural Networks with Affine Latent Parameterizations — arXiv:2608.31157},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.31157},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_52674aa48908; sha256 0ec535a25f2998ce53656f4e1d33582e7a09519babcffb6cff1c86035dbce112},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.31157. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Many parameter-efficient methods generate the parameters of a large neural network from a low-dimensional latent representation. Given an architecture $\Phi$ with $P_\Phi$ parameter slots, we write $\boldsymbol{\theta}_f=\mathcal{G}(\boldsymbol{\xi}_f)$, where $\mathcal{G}\colon\mathbb{R}^M\to\mathbb{R}^{P_\Phi}$ is a parameter generator and $\boldsymbol{\xi}_f\in\mathbb{R}^M$ is a latent representation of the target function $f$. The architecture $\Phi$ and the generator $\mathcal{G}$ are shared across the entire target class, while each target $f$ is represented by its own latent vector $\boldsymbol{\xi}_f$, with $\Phi_{\mathcal{G}(\boldsymbol{\xi}_f)}$ approximating $f$. This framework encompasses hypernetworks, low-dimensional parameterizations, parameter-efficient adaptation, and model compression. Understanding the tradeoff between the latent dimension $M$ and the network budget $P$ is therefore fundamental to characterizing the expressive efficiency of these methods. We study this tradeoff for affine generators and fully connected ReLU architectures. More precisely, optimizing jointly over architectures $\Phi$ satisfying $P_\Phi\leq P$ and affine generators $\mathcal{G}:\mathbb{R}^M\to \mathbb{R}^{P_\Phi}$, we prove that the optimal worst-case uniform approximation error over the unit ball of $\alpha$-Hölder functions on $[0,1]^d$, where $0<\alpha\leq1$, has the sharp order $ \bigl(P\min\{M,P\}\bigr)^{-\alpha/d}. $ In particular, our result shows that even a fixed-dimensional latent space suffices to achieve vanishing approximation error as the network budget increases.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2608.31157 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T08:13:26.870Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).