Metraipolis › archive › dc_e1ba0c1dc992
Large-scale factor analysis shows machine intelligence is only partially interpretable — arXiv:2609.36515
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36515
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:25:58.156Z
- SHA-256 of the archived text
290dba9a9f5eff30e3191fc94eee1a351f6f70f8a4c09707e6733a91adab4d91- Stable URL of this page
https://metraipolis.com/documento/dc_e1ba0c1dc992
How to cite
Large-scale factor analysis shows machine intelligence is only partially interpretable — arXiv:2609.36515 — arXiv (en), https://arxiv.org/abs/2609.36515 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e1ba0c1dc992 (Metraipolis archive dc_e1ba0c1dc992, sha256 290dba9a9f5eff30e3191fc94eee1a351f6f70f8a4c09707e6733a91adab4d91)
BibTeX
@misc{dce1ba0c1dc992,
title = {Large-scale factor analysis shows machine intelligence is only partially interpretable — arXiv:2609.36515},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36515},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e1ba0c1dc992; sha256 290dba9a9f5eff30e3191fc94eee1a351f6f70f8a4c09707e6733a91adab4d91},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36515. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
A common assumption in language model development is that cognitive abilities are organized around a general, domain-free intelligence factor, like fluid intelligence in humans. This assumption is rarely tested directly, and prior attempts have done so only at a much smaller scale. We take a latent variable approach to intelligence in language models, similar to how psychometricians study psychological constructs. Performance in every specific problem set is influenced by a domain-specific and a domain-agnostic latent factor. Using factor analysis as a dimension-reduction technique, we analyzed 13,251 published evaluation scores covering 1,618 language models across 456 different text-only benchmarks. Due to the super-sparse nature of the dataset, we triangulate our analysis across different data densifiers and imputation methods. A robust pattern across different modes of bias is that 1. A general intelligence factor accounts for 70.8% of variance in model performance at our most generous estimate, and far less than that in most of our solutions, 2. Content-similar benchmarks do not necessarily cluster together, and 3. The $g$ factor is not dominated by any common theme, and there is a lack of evidence that it is well-proxied by standard "intelligence" benchmarks. Our findings go against current endeavors of defining, identifying, and targeting general intelligence as a tangible construct in language model development. This leaves the strategy of targeting a single conceptual ability without support, since the first-order abilities it would have to reach are often partially idiosyncratic and not identifiable in practice. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Neurons and Cognition (q-bio.NC). Pagina del lavoro: https://arxiv.org/abs/2609.36515 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T15:54:53.850Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).