Metraipolis › archive › dc_bc15059ddfb1

Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference — arXiv:2608.10288

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.10288
License (as archived)
CC0-1.0
Archived
2026-10-19T06:14:26.915Z
SHA-256 of the archived text
ef96e157fbff12519e4dd9e5ec7fdfc0c46bd16dbb99112945c5d9b2308506c4
Stable URL of this page
https://metraipolis.com/documento/dc_bc15059ddfb1

How to cite

Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference — arXiv:2608.10288 — arXiv (en), https://arxiv.org/abs/2608.10288 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bc15059ddfb1 (Metraipolis archive dc_bc15059ddfb1, sha256 ef96e157fbff12519e4dd9e5ec7fdfc0c46bd16dbb99112945c5d9b2308506c4)
BibTeX
@misc{dcbc15059ddfb1,
  title = {Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference — arXiv:2608.10288},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.10288},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bc15059ddfb1; sha256 ef96e157fbff12519e4dd9e5ec7fdfc0c46bd16dbb99112945c5d9b2308506c4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.10288. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The Large Language Model from Power Law Decoder Representations (PLDR-LLM) and its attention, Power Law Graph Attention (PLGA), replace the fixed bilinear form of scaled dot-product attention (SDPA) with a learned, input-generated bilinear operator $G_{LM}$, built from a positive tensor $A_{LM}$ by elementwise power laws. The architecture is fully specified, verified against pinned reference releases; claims are labeled theorem, conditional theorem, measurement, or conjecture. Unconditionally: PLGA contains SDPA exactly at $G_{LM}=I$; $A_{LM}$ and $A_P$ are strictly entrywise positive, with Perron-Frobenius structure on $A_{LM}$; the DAG regularizer has the NOTEARS walk-counting form and positivity obstructs exact acyclicity; and, under nonresonance (satisfied by standard rotary frequencies), a commutant criterion identifies which operators preserve relative-position dependence. An inference-collapse theorem: exact input invariance of deductive outputs collapses inference to generalized SDPA with a constant operator. Measured invariance: relative fluctuations of $10^{-6}$ and below; perturbation bounds quantify but do not certify cached inference; the assembled proxy misses the decoding margin. A conditional three-stage mechanism (rotary twirl, concentration, row-map contraction) is measured on a released checkpoint. Blockwise training and scoring under the global Gram are stated with explicit target exposure; on tested samples, block and sequential scoring select identical answers and agree on the published TruthfulQA probability-mass metric within $5\times 10^{-5}$ per item. Self-organized criticality enters as a phenomenological framework with an intrinsic order parameter; open claims become falsifiable conjectures. Selected proof cores are machine-checked in Lean 4.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-12 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.10288 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T06:08:50.353Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).