Metraipolis › archive › dc_decd30524d8c
Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence — arXiv:2609.18612
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.18612
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T13:49:59.005Z
- SHA-256 of the archived text
644db06fb672b9a719d40cc975a39e5de419ce876a3bdb322e94adbc76558f48- Stable URL of this page
https://metraipolis.com/documento/dc_decd30524d8c
How to cite
Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence — arXiv:2609.18612 — arXiv (en), https://arxiv.org/abs/2609.18612 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_decd30524d8c (Metraipolis archive dc_decd30524d8c, sha256 644db06fb672b9a719d40cc975a39e5de419ce876a3bdb322e94adbc76558f48)
BibTeX
@misc{dcdecd30524d8c,
title = {Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence — arXiv:2609.18612},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.18612},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_decd30524d8c; sha256 644db06fb672b9a719d40cc975a39e5de419ce876a3bdb322e94adbc76558f48},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.18612. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We analyze the learned input-output behavior of GLU-based neurons in large language models (LLMs). We propose a simple analysis method: For each neuron, we compute the cosine similarities between its input (reading) and output (writing) weight vectors. In this scheme, a strong negative cosine similarity indicates the neuron weakens the direction it detects in the residual stream, so we call this a weakening neuron. This allows us to gain a number of novel insights. First, we show that nine different LLMs have similar patterns: weakening neurons appear mostly in late layers whereas their counterparts, (conditional) strengthening neurons, are frequent in early-middle layers. Second, we find that weakening neurons display surprising behavior: even though there are few, they activate often and have a large influence on model behavior. Third, weakening neurons have a strong effect on model output when gate values are negative -- which is surprising since negative gate values are not expected to encode functionality. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CL). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.18612 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T13:48:24.427Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).