Metraipolis › archive › dc_22ee35397fac

Encoding and Propagation of Grammatical Number in Large Language Models — arXiv:2611.07283

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.07283
License (as archived): CC0-1.0
Archived: 2026-11-06T14:42:18.536Z
SHA-256 of the archived text: b7eea73cc91278163d1a5afe61657598866025f63135c88ff57f128022d24772
Stable URL of this page: https://metraipolis.com/documento/dc_22ee35397fac

How to cite

Encoding and Propagation of Grammatical Number in Large Language Models — arXiv:2611.07283 — arXiv (en), https://arxiv.org/abs/2611.07283 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_22ee35397fac (Metraipolis archive dc_22ee35397fac, sha256 b7eea73cc91278163d1a5afe61657598866025f63135c88ff57f128022d24772)

BibTeX

@misc{dc22ee35397fac,
  title = {Encoding and Propagation of Grammatical Number in Large Language Models — arXiv:2611.07283},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.07283},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_22ee35397fac; sha256 b7eea73cc91278163d1a5afe61657598866025f63135c88ff57f128022d24772},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07283. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Sparse autoencoders (SAEs) have become an increasingly popular means of translating Large Language Model (LLM) hidden state vectors into a human-comprehensible list of features or symbols. As such, SAEs may provide insight into an LLM's knowledge representation and internal ``mental'' grammar - topics of pragmatic interest for mechanistic interpretation of AI behavior and of theoretical interest for the study of in silico representations of human language. Language models with fewer than 100M parameters, trained on small corpora, nonetheless exhibit strong grammatical competence and discourse coherence, despite producing content of limited reliability. We use SAEs that we trained on one such model (Pythia-70M) to conduct an in-depth examination of how a single grammatical phenomenon, number agreement, is represented and processed in an LLM. We demonstrate the mechanism and the methodology by which SAEs can be used to find and propagate ``plural noun'', ``plural matrix subject'', and ``generate a plural verb'' features which cause the LLM to produce plural verb forms such as ``are'', ``do'', or ``have'' instead of ``is'', ``does'', and ``has'' in response to prompts such as ``the men in the park''. We show that intervening to remove these features from the hidden states of plural subject noun tokens causes the model to produce singular verbs instead, and vice versa. We also show that corresponding ``singular noun'' and ``generate a singular verb'' features are not as effective for this task. This asymmetry suggests that Pythia-70m makes use of a privative representation of grammatical number in English. These findings replicated with larger LLMs Gemma-2-2b and Llama-3.1-8b yielded similar results. Our main contribution is a novel approach to the use of SAEs to understanding how LLMs encode human language that addresses a number of technical and methodological challenges.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.07283 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:18:42.672Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).