Metraipolis › archive › dc_ad0aaffd7562

Early Learning Shapes Later Directions Of Representation Change In Continual Learning — arXiv:2609.36081

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.36081
License (as archived)
CC0-1.0
Archived
2026-10-07T17:28:41.401Z
SHA-256 of the archived text
231f9031301fa05b0834768978186e8ea8579674f54fee72275e7798b9263a7c
Stable URL of this page
https://metraipolis.com/documento/dc_ad0aaffd7562

How to cite

Early Learning Shapes Later Directions Of Representation Change In Continual Learning — arXiv:2609.36081 — arXiv (en), https://arxiv.org/abs/2609.36081 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ad0aaffd7562 (Metraipolis archive dc_ad0aaffd7562, sha256 231f9031301fa05b0834768978186e8ea8579674f54fee72275e7798b9263a7c)
BibTeX
@misc{dcad0aaffd7562,
  title = {Early Learning Shapes Later Directions Of Representation Change In Continual Learning — arXiv:2609.36081},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.36081},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ad0aaffd7562; sha256 231f9031301fa05b0834768978186e8ea8579674f54fee72275e7798b9263a7c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36081. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Representations continually change as a network learns new tasks. We ask whether early representational changes naturally form a geometric structure that continues to shape later learning. We identify a low-dimensional subspace of early representation drift, which we call a scaffold, and test whether it is reused across subsequent tasks. Across four pretrained visual encoders and two datasets, later representational changes consistently favor this early-defined subspace over matched random alternatives. This reuse is history-dependent: when networks experience different early tasks but identical later training inputs, each network preferentially reuses the scaffold induced by its own learning history. The same preference appears in individual optimizer updates, even though the network's dominant local response directions shift away from the original scaffold. Finally, constraining motion within the scaffold slows new-task acquisition more than matched random constraints, while effects on old-task retention are less consistent. In summary, these results suggest that early experience leaves a persistent geometric imprint on how neural networks adapt to future this http URL is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.36081 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T16:59:48.097Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).