Metraipolis › archive › dc_21440cfa23fd
CruxBench: A Benchmark of Information Discovery — arXiv:2609.35879
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.35879
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:24:54.489Z
- SHA-256 of the archived text
f75cc55d3c7f71efbe3e91ec51f777d71683ea28272a9d542799d375c519ae47- Stable URL of this page
https://metraipolis.com/documento/dc_21440cfa23fd
How to cite
CruxBench: A Benchmark of Information Discovery — arXiv:2609.35879 — arXiv (en), https://arxiv.org/abs/2609.35879 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_21440cfa23fd (Metraipolis archive dc_21440cfa23fd, sha256 f75cc55d3c7f71efbe3e91ec51f777d71683ea28272a9d542799d375c519ae47)
BibTeX
@misc{dc21440cfa23fd,
title = {CruxBench: A Benchmark of Information Discovery — arXiv:2609.35879},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.35879},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_21440cfa23fd; sha256 f75cc55d3c7f71efbe3e91ec51f777d71683ea28272a9d542799d375c519ae47},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35879. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Benchmarks for large language models (LLMs) typically evaluate the accuracy of answers against fixed reference labels. But a central step in many complex real-world tasks is identifying which questions are worth asking in the first place: decomposing a difficult problem into subquestions -- which we call cruxes -- whose answers provide key steps on the path toward solving the target problem. To evaluate this capability of information discovery, we introduce CruxBench, a benchmark that grades LLM-generated questions by their Value of Information (VOI): how much a model-proposed crux updates beliefs about a target forecasting question. CruxBench enjoys a rare combination of three key properties: it is (1) contamination-resistant by construction, since ground truth is generated by future world events; (2) open-ended, admitting unbounded and complex text-based submissions rather than one correct numeric answer; and (3) grounded, with informativeness measured against quantified changes in real-world beliefs. We evaluate a diverse set of eight models on 293 target forecasting questions and find that VOI correlates highly with independent measures of model capability (r=0.90) and captures cruxes' usefulness for answering target questions. However, information discovery remains challenging even for frontier LLMs, which only narrowly outperform a random-timing baseline. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.35879 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T15:54:53.730Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).