Metraipolis › archive › dc_decd30524d8c

Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence — arXiv:2609.18612

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.18612
License (as archived)
CC0-1.0
Archived
2026-10-15T13:49:59.005Z
SHA-256 of the archived text
644db06fb672b9a719d40cc975a39e5de419ce876a3bdb322e94adbc76558f48
Stable URL of this page
https://metraipolis.com/documento/dc_decd30524d8c

How to cite

Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence — arXiv:2609.18612 — arXiv (en), https://arxiv.org/abs/2609.18612 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_decd30524d8c (Metraipolis archive dc_decd30524d8c, sha256 644db06fb672b9a719d40cc975a39e5de419ce876a3bdb322e94adbc76558f48)
BibTeX
@misc{dcdecd30524d8c,
  title = {Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence — arXiv:2609.18612},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.18612},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_decd30524d8c; sha256 644db06fb672b9a719d40cc975a39e5de419ce876a3bdb322e94adbc76558f48},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.18612. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We analyze the learned input-output behavior of GLU-based neurons in large language models (LLMs). We propose a simple analysis method: For each neuron, we compute the cosine similarities between its input (reading) and output (writing) weight vectors. In this scheme, a strong negative cosine similarity indicates the neuron weakens the direction it detects in the residual stream, so we call this a weakening neuron. This allows us to gain a number of novel insights. First, we show that nine different LLMs have similar patterns: weakening neurons appear mostly in late layers whereas their counterparts, (conditional) strengthening neurons, are frequent in early-middle layers. Second, we find that weakening neurons display surprising behavior: even though there are few, they activate often and have a large influence on model behavior. Third, weakening neurons have a strong effect on model output when gate values are negative -- which is surprising since negative gate values are not expected to encode functionality.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.18612 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:48:24.427Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).