Metraipolis › archive › dc_2241cd86ef4e

Do ResNets Route? Sparse Interaction Experts in Residual Networks — arXiv:2610.02907

Source: arXiv
Original: https://arxiv.org/abs/2610.02907
License (as archived): CC0-1.0
Archived: 2026-10-05T23:10:00.032Z
SHA-256 of the archived text: b377dab8a65f96140757d845a4b0918ad65f412dc460009452da62ec2e52350f
Stable URL of this page: https://metraipolis.com/documento/dc_2241cd86ef4e

How to cite

Do ResNets Route? Sparse Interaction Experts in Residual Networks — arXiv:2610.02907 — arXiv, https://arxiv.org/abs/2610.02907 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2241cd86ef4e (Metraipolis archive dc_2241cd86ef4e, sha256 b377dab8a65f96140757d845a4b0918ad65f412dc460009452da62ec2e52350f)

BibTeX

@misc{dc2241cd86ef4e,
  title = {Do ResNets Route? Sparse Interaction Experts in Residual Networks — arXiv:2610.02907},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.02907},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2241cd86ef4e; sha256 b377dab8a65f96140757d845a4b0918ad65f412dc460009452da62ec2e52350f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02907. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Residual networks execute every block for every input, yet their functional contributions need not be input independent. We formulate a trained ResNet as a set function over binary residual-branch masks and apply Möbius inversion to decompose its output exactly into individual residual corrections and higher-order interactions. For smooth residual stacks, we show that each fixed $k$-way interaction scales as $\mathcal{O}(\lambda^k)$ under residual scaling. Exhaustive analysis of ImageNet-pretrained ResNet-18 and ResNet-34 reveals that interaction mass peaks at orders five and ten, respectively, rather than at low orders. Reducing the residual scale shifts both spectra toward lower orders, but also changes model predictions. The interaction coefficients are concentrated in magnitude but not hard sparse, and prediction-preserving sparsity weakens with depth. Crucially, the dominant interactions vary across inputs and predicted classes around a shared global core, while their overall complexity changes little with sample difficulty. These results show that dense ResNets implement an implicit form of soft routing: every block is executed, but different inputs rely on different residual interaction experts. Routing can therefore emerge at the level of functional contribution without an explicit router or sparse execution.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.02907 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T23:01:38.652Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).