Metraipolis › archive › dc_3781dc4fc408

An Exploratory Replica-Overlap Probe of the Grokking Transition — arXiv:2609.25634

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.25634
License (as archived)
CC0-1.0
Archived
2026-10-08T13:46:34.199Z
SHA-256 of the archived text
3ad788e95dcfb4b55dc2f55ba3df6a87f723c145af1078d4b4d1b405a62774ed
Stable URL of this page
https://metraipolis.com/documento/dc_3781dc4fc408

How to cite

An Exploratory Replica-Overlap Probe of the Grokking Transition — arXiv:2609.25634 — arXiv (en), https://arxiv.org/abs/2609.25634 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3781dc4fc408 (Metraipolis archive dc_3781dc4fc408, sha256 3ad788e95dcfb4b55dc2f55ba3df6a87f723c145af1078d4b4d1b405a62774ed)
BibTeX
@misc{dc3781dc4fc408,
  title = {An Exploratory Replica-Overlap Probe of the Grokking Transition — arXiv:2609.25634},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.25634},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3781dc4fc408; sha256 3ad788e95dcfb4b55dc2f55ba3df6a87f723c145af1078d4b4d1b405a62774ed},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25634. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We trained 64 independently seeded networks in four configurations, continuing each to sustained convergence or a 40,000-epoch ceiling. We then asked whether an RSB-inspired distribution of pairwise weight overlaps changes across the grokking transition. It is the alignment step, not the overlap statistic, that determines what this registered probe can report. The registered implementation permutes hidden units without the corresponding bias and head-internal permutations and therefore does not preserve the network function. Every q_wt value computed through this alignment inherits the defect; q_fn does not, because it is computed from predictions of the unpermuted models. The numerical-precision requirement also failed, and an audit found protocol deviations. Consequently, the pre-registered rule gives no verdict: registered outcome UNDETERMINED (reason code C0_INSTRUMENT_INVALID). These data provide neither a confirmatory null nor a validated reading of the Parisi order parameter. Only frac40 cleared the 12/16 checkpoint-completeness requirement. For this configuration, a post-hoc criterion applied to the same data gave a Hartigan-dip interval containing zero (95% CI for Delta dip = [-0.017, 0.034]), whereas the overlap standard deviation increased by a factor of about 5.6. A post-hoc calibration assigns the dip test zero power at the simulated separations; the interval is therefore uninformative, not evidence of no change. The standard-deviation ratio is the only statistic here with power at the observed effect. Ensemble loss was near-flat only under the pre-specified 1% threshold. Finally, grokking rates of 0/16, 11/16 and 16/16 remain descriptive because train fraction is confounded with split identity.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Disordered Systems and Neural Networks (cond-mat.dis-nn); Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.25634 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T13:18:38.247Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).