Metraipolis › archive › dc_01ec23cc0094
System1-Arena: Which Decisions Can Be Left to System 1? — arXiv:2610.21244
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.21244
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T09:12:03.977Z
- SHA-256 of the archived text
80b0cb81088f3bc23d8f592955631487724f87e8801b9890e00d9b0a95d93226- Stable URL of this page
https://metraipolis.com/documento/dc_01ec23cc0094
How to cite
System1-Arena: Which Decisions Can Be Left to System 1? — arXiv:2610.21244 — arXiv (en), https://arxiv.org/abs/2610.21244 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_01ec23cc0094 (Metraipolis archive dc_01ec23cc0094, sha256 80b0cb81088f3bc23d8f592955631487724f87e8801b9890e00d9b0a95d93226)
BibTeX
@misc{dc01ec23cc0094,
title = {System1-Arena: Which Decisions Can Be Left to System 1? — arXiv:2610.21244},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.21244},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_01ec23cc0094; sha256 80b0cb81088f3bc23d8f592955631487724f87e8801b9890e00d9b0a95d93226},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21244. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) often make discrete decisions, such as flagging phishing or escalating to a human. System-1 models, such as Jev and the OpenAI Decisions API, return probabilities over options in a single forward pass, while System-2 approaches generate reasoning steps. Which decisions can System-1 models handle, and which require generative reasoning? We introduce System1-Arena, a benchmark that gives all models the same decision requests and scores their probabilities. It contains 92k questions across 105 tasks from 268 public sources, in four groups: text classification and knowledge, applied decisions, evaluation and control, and agent and action. It also includes 138 closed-loop environments, from web shopping and SWE-bench coding control to robotics, games, and driving; in real-time mode, the world evolves while a model decides. We evaluate 30 models: 7 closed LLMs, 13 open LLMs read at the answer token, and 10 System-1 models. The largest System-1 models match or exceed fast closed LLMs in non-agentic accuracy, with better-calibrated probabilities at one-quarter of the latency. A reasoning LLM is the most accurate, at about 70x the cost of a fast LLM. In closed loops, large open LLMs succeed most often (up to 32.9%); small System-1 models rarely succeed. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.21244 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:44:17.924Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).