Metraipolis › archive › dc_8bf33cedd74c

Where Grokking Happens: Distributed Utility and Fourier Recoding Without a Module Switch — arXiv:2609.17571

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.17571
License (as archived): CC0-1.0
Archived: 2026-10-15T15:04:19.274Z
SHA-256 of the archived text: bfc43043c43166e2063d6a92e797f148c89bf92bc5721eaac5010428afdb4d97
Stable URL of this page: https://metraipolis.com/documento/dc_8bf33cedd74c

How to cite

Where Grokking Happens: Distributed Utility and Fourier Recoding Without a Module Switch — arXiv:2609.17571 — arXiv (en), https://arxiv.org/abs/2609.17571 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8bf33cedd74c (Metraipolis archive dc_8bf33cedd74c, sha256 bfc43043c43166e2063d6a92e797f148c89bf92bc5721eaac5010428afdb4d97)

BibTeX

@misc{dc8bf33cedd74c,
  title = {Where Grokking Happens: Distributed Utility and Fourier Recoding Without a Module Switch — arXiv:2609.17571},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.17571},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8bf33cedd74c; sha256 bfc43043c43166e2063d6a92e797f148c89bf92bc5721eaac5010428afdb4d97},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17571. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Where in a Transformer is the change from memorization to generalization functionally expressed? We introduce Transition Games--behavior-aligned exact activation games with paired non-generalizing controls--and find distributed utility gain with a prospective block-0 attention bias; selected degree-two modes account for 67--92% of its addition contrast across replacement games, and a disjoint exact path study confirms that block-1 MLP mediates more of their effect than all other tested downstream paths in 12/12 pairs. The sharper "MLP memorizes, attention generalizes" prediction instead reverses (-.331 bits/example at the memory anchor; 0/12 in the predicted direction), while routing onset, global rank collapse, and a prime-invariant architecture ridge also fail, identifying grokking here as spectral recoding of an existing distributed circuit rather than a module switch.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.17571 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T14:08:19.046Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).