Metraipolis › archive › dc_19cb78f025ee
Towards an Optimal Control Foundation of Early Exits - Turnpikes and ResNets — arXiv:2611.05120
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.05120
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T21:51:55.545Z
- SHA-256 of the archived text
141d9b1896d4f2af1e0592e1ba3971125640d4ddd37d5134b13759b868c34d42- Stable URL of this page
https://metraipolis.com/documento/dc_19cb78f025ee
How to cite
Towards an Optimal Control Foundation of Early Exits - Turnpikes and ResNets — arXiv:2611.05120 — arXiv (en), https://arxiv.org/abs/2611.05120 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_19cb78f025ee (Metraipolis archive dc_19cb78f025ee, sha256 141d9b1896d4f2af1e0592e1ba3971125640d4ddd37d5134b13759b868c34d42)
BibTeX
@misc{dc19cb78f025ee,
title = {Towards an Optimal Control Foundation of Early Exits - Turnpikes and ResNets — arXiv:2611.05120},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.05120},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_19cb78f025ee; sha256 141d9b1896d4f2af1e0592e1ba3971125640d4ddd37d5134b13759b868c34d42},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05120. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The feed-forward propagation through neural networks is crucial for training and inference. To trade off efficiency and accuracy, it has been proposed to consider intermediate outputs of hidden states as data-dependent regularization during training and, ultimately, as early-exits during inference. In this work, we analyze the joint optimization of loss functions computed for the output layer and intermediate layers using optimal control theory. We prove that ResNets trained in this joint fashion exhibit the so-called turnpike phenomenon, this implies that the number of layers required to reach a prescribed loss level is bounded from above independently of the overall network depth. This induces a similarity property across the internal representations of early-exiting networks of varying depths. We connect the turnpike to the overthinking phenomenon, and provide theoretical insights into the success of early-exit criteria derived from softmax probabilities, specifically entropy and max-probability. In a range of experiments, we demonstrate the existence and effect of turnpikes and the resulting similarity in the early-exit performance for ResNets of varying depths. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Systems and Control (eess.SY). Pagina del lavoro: https://arxiv.org/abs/2611.05120 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T21:38:49.751Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).