Metraipolis › archive › dc_016dd67ac337

Towards an Optimal Control Foundation of Early Exits - Turnpikes and ResNets — arXiv:2611.05120

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.05120
License (as archived): CC0-1.0
Archived: 2026-11-04T18:54:23.132Z
SHA-256 of the archived text: de6c7cced3f873d32ad4f679da0a293100c12c38bbe66e5644b9f2a78b627ae5
Stable URL of this page: https://metraipolis.com/documento/dc_016dd67ac337

How to cite

Towards an Optimal Control Foundation of Early Exits - Turnpikes and ResNets — arXiv:2611.05120 — arXiv (en), https://arxiv.org/abs/2611.05120 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_016dd67ac337 (Metraipolis archive dc_016dd67ac337, sha256 de6c7cced3f873d32ad4f679da0a293100c12c38bbe66e5644b9f2a78b627ae5)

BibTeX

@misc{dc016dd67ac337,
  title = {Towards an Optimal Control Foundation of Early Exits - Turnpikes and ResNets — arXiv:2611.05120},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05120},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_016dd67ac337; sha256 de6c7cced3f873d32ad4f679da0a293100c12c38bbe66e5644b9f2a78b627ae5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05120. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The feed-forward propagation through neural networks is crucial for training and inference. To trade off efficiency and accuracy, it has been proposed to consider intermediate outputs of hidden states as data-dependent regularization during training and, ultimately, as early-exits during inference. In this work, we analyze the joint optimization of loss functions computed for the output layer and intermediate layers using optimal control theory. We prove that ResNets trained in this joint fashion exhibit the so-called turnpike phenomenon, this implies that the number of layers required to reach a prescribed loss level is bounded from above independently of the overall network depth. This induces a similarity property across the internal representations of early-exiting networks of varying depths. We connect the turnpike to the overthinking phenomenon, and provide theoretical insights into the success of early-exit criteria derived from softmax probabilities, specifically entropy and max-probability. In a range of experiments, we demonstrate the existence and effect of turnpikes and the resulting similarity in the early-exit performance for ResNets of varying depths.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Systems and Control (eess.SY).
Pagina del lavoro: https://arxiv.org/abs/2611.05120 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T18:35:37.478Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).