Metraipolis › archive › dc_a6a17e3522ff

CogBattery: Human-Referenced Evaluation of Cognitive Task Behavior and Structure in Language Models — arXiv:2607.24999

Source
arXiv (en)
Original
https://arxiv.org/abs/2607.24999
License (as archived)
CC0-1.0
Archived
2026-11-03T19:07:03.206Z
SHA-256 of the archived text
57bed36ca4be1e30eefcd0da4a005c438486aa9f196249dd61a7a98381167519
Stable URL of this page
https://metraipolis.com/documento/dc_a6a17e3522ff

How to cite

CogBattery: Human-Referenced Evaluation of Cognitive Task Behavior and Structure in Language Models — arXiv:2607.24999 — arXiv (en), https://arxiv.org/abs/2607.24999 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a6a17e3522ff (Metraipolis archive dc_a6a17e3522ff, sha256 57bed36ca4be1e30eefcd0da4a005c438486aa9f196249dd61a7a98381167519)
BibTeX
@misc{dca6a17e3522ff,
  title = {CogBattery: Human-Referenced Evaluation of Cognitive Task Behavior and Structure in Language Models — arXiv:2607.24999},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2607.24999},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a6a17e3522ff; sha256 57bed36ca4be1e30eefcd0da4a005c438486aa9f196249dd61a7a98381167519},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2607.24999. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Human cognitive tasks let us compare language models with humans under controlled conditions. Yet a model can reproduce the direction of a human effect while making different errors. CogBattery evaluates 54 open-weight language models on text adaptations of 13 cognitive paradigms, comparing each with human data from its source study. Like humans, models respond more to n-back lures than to controls, yet they make far more false alarms on ordinary non-targets. Most models' reported confidence separates correct from incorrect answers less well than humans' does, and most always choose the same wager, including models that classify accurately. These patterns span many model families. Across tasks, model scores share two dimensions. The first is closely tied to scale, yet larger models are not consistently closer to humans. Predefined cognitive groupings show no reliable predictive advantage over random groupings of the same size. These results show that matching human effect directions does not establish human-like behavior. CogBattery provides condition-level human references for testing errors and confidence reports in new models.

Tema degli esploratori: scienze della vita e salute (elenco arXiv q-bio.NC, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Neurons and Cognition (q-bio.NC).
Pagina del lavoro: https://arxiv.org/abs/2607.24999 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:23:22.754Z dalla pagina https://arxiv.org/list/q-bio.NC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).