Metraipolis › archive › dc_445f35fef172

Nonconformal Distortion and Local Orientation Reversal in the Fourier Circuit of a Modular-Addition Transformer — arXiv:2611.00414

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.00414
License (as archived): CC0-1.0
Archived: 2026-11-03T10:30:47.539Z
SHA-256 of the archived text: b3be57c14f261d38665a1aa53817c0284b3d57ddc11b0578714e6d00b511669c
Stable URL of this page: https://metraipolis.com/documento/dc_445f35fef172

How to cite

Nonconformal Distortion and Local Orientation Reversal in the Fourier Circuit of a Modular-Addition Transformer — arXiv:2611.00414 — arXiv (en), https://arxiv.org/abs/2611.00414 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_445f35fef172 (Metraipolis archive dc_445f35fef172, sha256 b3be57c14f261d38665a1aa53817c0284b3d57ddc11b0578714e6d00b511669c)

BibTeX

@misc{dc445f35fef172,
  title = {Nonconformal Distortion and Local Orientation Reversal in the Fourier Circuit of a Modular-Addition Transformer — arXiv:2611.00414},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00414},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_445f35fef172; sha256 b3be57c14f261d38665a1aa53817c0284b3d57ddc11b0578714e6d00b511669c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00414. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We study the local differential geometry of a Fourier-based modular-addition circuit in a one-layer transformer whose algorithm has previously been reverse-engineered. For each of the five key Fourier frequencies, we restrict the model's multilayer perceptron (MLP) contribution to a corresponding Fourier input plane and readout, and measure its directional Beltrami coefficient under continuous input perturbations. Whereas nondegenerate restrictions of ideal complex multiplication are conformal, the trained maps exhibit nonconformal distortion (median $|\mu|=0.304$--$0.446$), spatial organization in the Beltrami argument, and regions of local orientation reversal. The median across frequencies decreases from $0.939$ at initialization to $0.390$ after training. The key-frequency planes also show lower median distortion than most random-plane controls, although their distributions overlap and the coordinate frames differ in conditioning. Across 500 checkpoints in one training run, most of the aggregate reduction in Beltrami magnitude occurs before the principal increase in the measured Fourier-circuit formation. At the individual-input level, Beltrami magnitude shows negligible association with the correct-logit margin ($|\rho|\leq0.026$). Late-checkpoint measurements across six runs show comparable distortion for most selected frequencies. These results provide a quantitative characterization of local deformation in a frequency-projected component of an identified neural circuit, complementary to its algorithmic reconstruction.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.00414 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:29:48.442Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).