Metraipolis › archive › dc_66518afd3752

Complex-valued Phase-Coherent Transformers — arXiv:2609.22415

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.22415
License (as archived)
CC0-1.0
Archived
2026-10-09T07:06:30.161Z
SHA-256 of the archived text
874e72d2a33b9d0ad461cf149e0397847df9e657fd2fae74f7a2176f75e26703
Stable URL of this page
https://metraipolis.com/documento/dc_66518afd3752

How to cite

Complex-valued Phase-Coherent Transformers — arXiv:2609.22415 — arXiv (en), https://arxiv.org/abs/2609.22415 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_66518afd3752 (Metraipolis archive dc_66518afd3752, sha256 874e72d2a33b9d0ad461cf149e0397847df9e657fd2fae74f7a2176f75e26703)
BibTeX
@misc{dc66518afd3752,
  title = {Complex-valued Phase-Coherent Transformers — arXiv:2609.22415},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.22415},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_66518afd3752; sha256 874e72d2a33b9d0ad461cf149e0397847df9e657fd2fae74f7a2176f75e26703},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22415. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Complex-valued Transformers have inherited softmax attention over the raw complex inner product. Outside natively complex domains this standard form stays near chance, and no complex attention had been shown to correct it. We show that the match must be a scaled cosine score: L2-normalise queries and keys, so the score reads their cosine similarity and ignores their magnitudes, and hold that score at order-one scale. With this the same models train on four diagnostic tasks under two different gates; without the normalisation they stay at chance on ListOps and Needle under both gates and fall far below on the other two, and a normalised score placed at too small a scale fails as well. The resulting family of phase-coherent Transformers (\PCT) matches or exceeds the strongest real-valued baseline across long-range memory, positional retrieval, hierarchical reasoning, frequency-domain classification and physical complex signals; it shows no degradation up to depth 20; and its loss decreases log-linearly over a 61-fold range of parameters. A member of the family, complex screening combined with a phase-coherent recurrence, is the first genuinely complex-valued neural network to solve Path-X, with 91.6% of its trainable parameters complex-valued against 38.2% for S4. We record these as signs of generalisation not previously seen in complex-valued neural networks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.22415 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:58:09.193Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).