Metraipolis › archive › dc_7eaccbb689db

S1CogBench: What System One Decision Models Can and Cannot Do on Cognitive Experiments — arXiv:2611.00911

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.00911
License (as archived): CC0-1.0
Archived: 2026-11-03T10:26:05.621Z
SHA-256 of the archived text: 2f779e105296f2e5750888f46641fdfcf9b5dd67a1502e02608369631b2e5c9d
Stable URL of this page: https://metraipolis.com/documento/dc_7eaccbb689db

How to cite

S1CogBench: What System One Decision Models Can and Cannot Do on Cognitive Experiments — arXiv:2611.00911 — arXiv (en), https://arxiv.org/abs/2611.00911 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7eaccbb689db (Metraipolis archive dc_7eaccbb689db, sha256 2f779e105296f2e5750888f46641fdfcf9b5dd67a1502e02608369631b2e5c9d)

BibTeX

@misc{dc7eaccbb689db,
  title = {S1CogBench: What System One Decision Models Can and Cannot Do on Cognitive Experiments — arXiv:2611.00911},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00911},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7eaccbb689db; sha256 2f779e105296f2e5750888f46641fdfcf9b5dd67a1502e02608369631b2e5c9d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00911. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Language models are no longer used only to generate text. Software also asks them for decisions, in routing, guardrails, and automated gates. System One decision models, also called typed decision models, such as Jev and Laya, return one fast decision over the legal options, leaving no text to inspect. However, a fast, well-formed decision does not show that the model used the evidence. Cognitive experiments can test this, because they vary task demands and measure how behavior changes, yet existing cognitive benchmarks evaluate language and multimodal models. We introduce S1CogBench to ask what these models can and cannot do on 24 cognitive tasks, evaluating five open models and Jev and keeping conditions and scoring apart from disclosed adaptations. We find that decision models vary widely in reliability, acting well does not imply predicting humans, aggregate scores hide how decisions respond to conditions, and speed comes mainly from answering without generation. To our knowledge, S1CogBench is the first benchmark of System One decision models on established cognitive experiments, and it gives a shared test of whether their decisions respond to evidence, feedback, memory load, and future opportunities before software relies on them.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Neurons and Cognition (q-bio.NC).
Pagina del lavoro: https://arxiv.org/abs/2611.00911 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:19:50.039Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).