Metraipolis › archive › dc_510969a7136c

Behavioral Convergence Without Representational Convergence: Persistent Training-History Dependence in Neural Networks — arXiv:2609.37836

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.37836
License (as archived)
CC0-1.0
Archived
2026-10-07T17:31:47.943Z
SHA-256 of the archived text
459158434a82841d453586075a14d8b5609d9c04f83ed278c9b5dfbc42b4e22f
Stable URL of this page
https://metraipolis.com/documento/dc_510969a7136c

How to cite

Behavioral Convergence Without Representational Convergence: Persistent Training-History Dependence in Neural Networks — arXiv:2609.37836 — arXiv (en), https://arxiv.org/abs/2609.37836 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_510969a7136c (Metraipolis archive dc_510969a7136c, sha256 459158434a82841d453586075a14d8b5609d9c04f83ed278c9b5dfbc42b4e22f)
BibTeX
@misc{dc510969a7136c,
  title = {Behavioral Convergence Without Representational Convergence: Persistent Training-History Dependence in Neural Networks — arXiv:2609.37836},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37836},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_510969a7136c; sha256 459158434a82841d453586075a14d8b5609d9c04f83ed278c9b5dfbc42b4e22f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37836. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Neural networks trained toward the same final objective can reach similar predictive performance while retaining internal representations shaped by earlier training history. We study this effect using controlled sequential-training experiments in which paired convolutional networks start from identical weights, experience reversed task orders, and then receive the same deterministic common-relaxation distribution. Across 20 paired MNIST runs, 16 satisfy a predeclared behavioral-matching criterion, yet their matched representations retain a mean history score of 0.139 (95% bootstrap CI: 0.127-0.153) and approximately 3.1% prediction disagreement. Extending common relaxation to 50,000 optimizer updates does not erase the measured difference: across five paired seeds, the representation-history score remains 0.190 (95% bootstrap CI: 0.161-0.219) at the end of the measured horizon while the mean accuracy gap is only 0.18 percentage points. Fresh linear probes show that, with sufficient labeled data, the two histories retain practically equivalent linearly accessible class information. A same-label rotated-MNIST control reproduces the effect: all five paired seeds reach behavioral matching while retaining a mean representation-history score of 0.162. Finally, a matched-learning-rate ReLU-LeakyReLU control reduces the 50,000-update representation residue by 0.040 on average in all five paired seeds, providing directional evidence that activation-mediated plasticity contributes to the persistence of training-history effects. These results provide protocol-scoped evidence that behavioral convergence need not imply representational convergence and that optimization history can leave measurable internal traces after prolonged common training.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.37836 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T16:59:48.379Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).