Metraipolis › archive › dc_98100182cc8b
Dissecting Geometric Constraints in Grokking: Interaction-Dependent Acceleration in Modular Arithmetic — arXiv:2611.03156
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.03156
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T18:38:53.453Z
- SHA-256 of the archived text
dd24248cd5683d61e4a66c3ac7463d13497c202f76c217ff2bde300f17fcdca0- Stable URL of this page
https://metraipolis.com/documento/dc_98100182cc8b
How to cite
Dissecting Geometric Constraints in Grokking: Interaction-Dependent Acceleration in Modular Arithmetic — arXiv:2611.03156 — arXiv (en), https://arxiv.org/abs/2611.03156 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_98100182cc8b (Metraipolis archive dc_98100182cc8b, sha256 dd24248cd5683d61e4a66c3ac7463d13497c202f76c217ff2bde300f17fcdca0)
BibTeX
@misc{dc98100182cc8b,
title = {Dissecting Geometric Constraints in Grokking: Interaction-Dependent Acceleration in Modular Arithmetic — arXiv:2611.03156},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.03156},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_98100182cc8b; sha256 dd24248cd5683d61e4a66c3ac7463d13497c202f76c217ff2bde300f17fcdca0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03156. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Spherical transformer configurations can accelerate grokking through simultaneous modifications of several model properties. The present study investigated the contributions of these modifications in the published Sphere+Fourier configuration using 960 controlled modular-arithmetic runs. A complete factorial analysis varied residual projection, readout weight normalization, logit scale, and weight decay. Across four matched settings with weight decay, the successful-run mean crossing times without projection were 8-37 times those observed with projection. However, residual projection was insufficient to induce grokking without additional interventions. Various other unprojected configurations also generalized. Since projection altered the magnitude entering the readout, these comparisons could not separate geometry from output scale. A separate 100-run fixed-horizon study demonstrated faster first crossing for the complete configuration across five train/test partitions. Faster crossing did not necessarily correspond to greater persistence over extended training. Control experiments demonstrated rapid grokking without Fourier initialization. Task and depth sweeps also identified conditions in which the configuration failed to reach the criterion by 50K epochs. Accordingly, the observed acceleration depended on interactions among the components and did not extend uniformly across the tested tasks or depths. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Computational Complexity (cs.CC). Pagina del lavoro: https://arxiv.org/abs/2611.03156 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T18:12:54.617Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).