Metraipolis › archive › dc_e832315fcbc8

Increasing Width Allows Greedy Layer-wise Training to Rival End-to-End Backpropagation in Self-Supervised Learning — arXiv:2610.00753

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.00753
License (as archived)
CC0-1.0
Archived
2026-10-06T12:11:01.385Z
SHA-256 of the archived text
db7fef429a68eadef8ab00c60f1f340c9a29566430bf5ce19e2bb6d70d8ad720
Stable URL of this page
https://metraipolis.com/documento/dc_e832315fcbc8

How to cite

Increasing Width Allows Greedy Layer-wise Training to Rival End-to-End Backpropagation in Self-Supervised Learning — arXiv:2610.00753 — arXiv (en), https://arxiv.org/abs/2610.00753 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e832315fcbc8 (Metraipolis archive dc_e832315fcbc8, sha256 db7fef429a68eadef8ab00c60f1f340c9a29566430bf5ce19e2bb6d70d8ad720)
BibTeX
@misc{dce832315fcbc8,
  title = {Increasing Width Allows Greedy Layer-wise Training to Rival End-to-End Backpropagation in Self-Supervised Learning — arXiv:2610.00753},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.00753},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e832315fcbc8; sha256 db7fef429a68eadef8ab00c60f1f340c9a29566430bf5ce19e2bb6d70d8ad720},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.00753. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

End-to-end backpropagation has been the dominant mode of training in deep learning, allowing for the coordination of parameter updates across layers of a neural network. Prior studies have explored alternative -- and, in some cases, simpler -- training mechanisms, showing that they can sometimes achieve performance similar to backpropagation. However, the architectural conditions under which locally optimized networks, which avoid end-to-end backpropagation of error, can learn representations comparable to those learned through end-to-end training remain unclear. We aim to answer this question in the context of self-supervised learning, an important framework for large-scale pretraining in artificial intelligence. Here, we investigate how network width and depth affect the efficacy of greedy layer-wise and end-to-end self-supervised training in convolutional networks. We find that in wider networks, the benefits of end-to-end backpropagation over greedy layer-wise training shrink: in relatively shallow and very wide networks, we even observed higher performance in models trained with greedy layer-wise training. Subsequent analysis of the representations formed by these networks shows that very wide greedy-trained networks exhibit more favorable representational geometry than do networks trained end-to-end with backpropagation. This work shows that width can compensate for restricted credit assignment and identifies differences in representational geometry as a potential mechanism for their improved performance.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Neural and Evolutionary Computing (cs.NE); Neurons and Cognition (q-bio.NC).
Pagina del lavoro: https://arxiv.org/abs/2610.00753 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:28.108Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).