Metraipolis › archive › dc_8f39b335c2d4

Archetypal contexts in the neurons of language models: an architecture for intelligence? — arXiv:2611.07234

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.07234
License (as archived)
CC0-1.0
Archived
2026-11-06T14:42:14.660Z
SHA-256 of the archived text
39dda5b50b375783b64d1e74352d3d0abaddbb28688dc88a1460e733d8ed778f
Stable URL of this page
https://metraipolis.com/documento/dc_8f39b335c2d4

How to cite

Archetypal contexts in the neurons of language models: an architecture for intelligence? — arXiv:2611.07234 — arXiv (en), https://arxiv.org/abs/2611.07234 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8f39b335c2d4 (Metraipolis archive dc_8f39b335c2d4, sha256 39dda5b50b375783b64d1e74352d3d0abaddbb28688dc88a1460e733d8ed778f)
BibTeX
@misc{dc8f39b335c2d4,
  title = {Archetypal contexts in the neurons of language models: an architecture for intelligence? — arXiv:2611.07234},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.07234},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8f39b335c2d4; sha256 39dda5b50b375783b64d1e74352d3d0abaddbb28688dc88a1460e733d8ed778f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07234. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We validate our previous hypothesis that a language model holds relational patterns apart from the subject domains that instantiate them (Nordfors, 2026). These archetypal contexts are patterns of relations that people recognize across many fields, such as the tipping point or winner takes all, held once and applied across domains. We show this inside the model, Qwen3.8-27B. One set of neurons carries the archetype, say the tipping point; another carries the topic, say politics, ecology or cell biology; a third carries the language, such as English, Spanish or Chinese. Silence the tipping point's neurons, and the model no longer identifies the pattern but still knows what the text is about. Silence the topic's, and it loses the topic but still names the pattern. In the neuron maps the layers organize themselves into a sandwich: the outer layers carry languages, the layers inside them topics, the layers at the heart archetypes. The sandwich emerges spontaneously during pretraining and appears in every model we mapped. Archetypes are the architecture for analogy, and we strongly suggest that this structure of archetypes and topics is the mechanism of a language model's intelligence: we see it self-organize as the model is trained. The neurons, the texts and the code are released with this paper as its reproduction package, so that anyone can repeat the test inside the model.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.07234 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:18:42.669Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).