Metraipolis › archive › dc_64dbc283c470
Structure is not mechanism: high-gain gated-FFN rows across text and genomic foundation models — arXiv:2609.17599
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.17599
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T15:05:04.906Z
- SHA-256 of the archived text
4a71953299bc93a3468e56b3df84c4d7c83c2406b7885d1b6a95377b54ba6f73- Stable URL of this page
https://metraipolis.com/documento/dc_64dbc283c470
How to cite
Structure is not mechanism: high-gain gated-FFN rows across text and genomic foundation models — arXiv:2609.17599 — arXiv (en), https://arxiv.org/abs/2609.17599 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_64dbc283c470 (Metraipolis archive dc_64dbc283c470, sha256 4a71953299bc93a3468e56b3df84c4d7c83c2406b7885d1b6a95377b54ba6f73)
BibTeX
@misc{dc64dbc283c470,
title = {Structure is not mechanism: high-gain gated-FFN rows across text and genomic foundation models — arXiv:2609.17599},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.17599},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_64dbc283c470; sha256 4a71953299bc93a3468e56b3df84c4d7c83c2406b7885d1b6a95377b54ba6f73},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17599. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
A small number of unusually high-gain parameters can exert disproportionate effects in transformer language models, but whether analogous structures recur in genomic foundation models and whether structural geometry determines functional importance remains unknown. We analyzed high-gain rows in gated feed-forward networks across text and genomic foundation models, including a frozen 22-model causal census. Computing an associated bilinear weight operator exactly, without a diagonal approximation, we tested whether structural extremeness is a transferable mechanism. Activation-derived candidates were functionally enriched relative to random and top-norm same-layer controls, yet neither spectral concentration nor operator magnitude predicted causal effect size, and these associations vanished within the endpoint-homogeneous text-decoder subset. A within-layer sweep of 36 rows in one genomic and one text decoder resolved this into two regimes: below the detector's acceptance threshold the ratio carried no positive information about causal damage, whereas above it the ratio ordered rows strongly but did not grade severity as a dose-response. The same sweep revealed a second individually catastrophic row invisible to a one-candidate-per-model census, and non-additive damage among co-located critical rows. Case studies showed divergent causal organizations: a robust super-additive pair interaction in DNABERT-2, and in GENERator a sharply position-localized dependence in which preserving or restoring the row's beginning-of-sequence contribution rescued essentially all native-loss damage. High-gain gated-FFN rows are therefore a recurrent architectural phenotype whose structural prominence acts as an enrichment signal, not a calibrated measure of functional criticality or a specification of causal organization. Enrichment is general, but the mechanism is model-specific. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.LG). Categorie arXiv: Genomics (q-bio.GN) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.17599 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T14:08:19.190Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).