Metraipolis › archive › dc_7dc4ffb48bdb

CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition — arXiv:2609.21259

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.21259
License (as archived): CC0-1.0
Archived: 2026-10-10T01:01:43.244Z
SHA-256 of the archived text: 43d173b095343e576f65fd0bfc283e843817df093954afb571dc9ebc9fb0039c
Stable URL of this page: https://metraipolis.com/documento/dc_7dc4ffb48bdb

How to cite

CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition — arXiv:2609.21259 — arXiv (en), https://arxiv.org/abs/2609.21259 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7dc4ffb48bdb (Metraipolis archive dc_7dc4ffb48bdb, sha256 43d173b095343e576f65fd0bfc283e843817df093954afb571dc9ebc9fb0039c)

BibTeX

@misc{dc7dc4ffb48bdb,
  title = {CogGym: Towards Large-Scale Comparative Evaluation of Human and Machine Cognition — arXiv:2609.21259},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.21259},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7dc4ffb48bdb; sha256 43d173b095343e576f65fd0bfc283e843817df093954afb571dc9ebc9fb0039c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.21259. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Understanding and modeling human intelligence are parallel goals shared by artificial intelligence (AI) and cognitive science. As AI systems grow increasingly capable, in what ways do model responses resemble human responses, and where do they systematically diverge? The sheer breadth and diversity of the tasks humans can perform and think about pose a challenge for scalable and rigorous comparison between humans and models. We introduce CogGym, a scalable, unified framework grounded in cognitive science for systematically comparing model and human behavior on matched experimental trials. CogGym uses a semi-automated, human-in-the-loop pipeline to standardize diverse experimental paradigms into a task-agnostic Experiment Markup Language (EML), enabling reproducible and faithful comparison at scale. For initial release, we curate and standardize 258 cognitive experiments from 100 papers that focuses on human commonsense reasoning, and evaluate 50 large language models against human responses. We find a clear scaling trend where larger and more recent AI models better reproduce human judgments. Yet AI models' improvement on such common reasoning tasks is considerably slower than the gains observed on formal-reasoning benchmarks like math and coding, and model--human fit remains well below human splithalf reliability ($R^2 = 0.93$ on text, $0.95$ on image, and $0.92$ on video) with the best models achieving $R^2 = 0.59$ on text, $0.58$ on image, and $0.43$ on video experiments. We intend for CogGym to provide a living evaluation framework that continually incorporates new cognitive science experiments to characterize where model behavior resembles human behavior, where it systematically diverges, and how those patterns change as models and experiments evolve.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-21 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.21259 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T00:23:42.441Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).