Metraipolis › archive › dc_7a68679e76dd

Interpretable Humans, Alien LLMs: Expert Analysis of Latent Structures in Assessment Responses — arXiv:2608.17810

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.17810
License (as archived)
CC0-1.0
Archived
2026-10-18T14:33:42.016Z
SHA-256 of the archived text
df4707e9d77755567215c02d84e927926a68332e8a88be7f9c9a48f503b41aaa
Stable URL of this page
https://metraipolis.com/documento/dc_7a68679e76dd

How to cite

Interpretable Humans, Alien LLMs: Expert Analysis of Latent Structures in Assessment Responses — arXiv:2608.17810 — arXiv (en), https://arxiv.org/abs/2608.17810 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7a68679e76dd (Metraipolis archive dc_7a68679e76dd, sha256 df4707e9d77755567215c02d84e927926a68332e8a88be7f9c9a48f503b41aaa)
BibTeX
@misc{dc7a68679e76dd,
  title = {Interpretable Humans, Alien LLMs: Expert Analysis of Latent Structures in Assessment Responses — arXiv:2608.17810},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.17810},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7a68679e76dd; sha256 df4707e9d77755567215c02d84e927926a68332e8a88be7f9c9a48f503b41aaa},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17810. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The evaluation of large language models (LLMs) relies heavily on human-designed assessments, implicitly assuming that AI and humans employ similar underlying cognitive constructs. Challenging this assumption, we investigate whether the latent factors governing LLM performance carry the same substantive, human-interpretable meaning as the cognitive constructs governing human learners. Using responses from humans and six LLMs across quantitative reasoning and chemistry assessments, we conducted Exploratory Factor Analysis (EFA) separately for both groups. Subject-Matter Experts (SMEs) then blindly evaluated the resulting factor graphs to ascribe pedagogical meaning to the emerged constructs. SMEs successfully interpreted most of the human-derived factors. Conversely, they could not ascribe meaning to any LLM-derived factors in quantitative reasoning and interpreted only half of the LLM factors in chemistry. By combining data-driven EFA with blind expert interpretation, this framework shows that LLMs frequently operate on statistically opaque mechanisms distinct from human reasoning.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2608.17810 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T14:27:54.459Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).