Metraipolis › archive › dc_4783e8968809

OpenFrontiers: A Collection of Open Research Problems and a Benchmark for Scientific Rediscovery — arXiv:2610.37739

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.37739
License (as archived)
CC0-1.0
Archived
2026-10-31T06:07:25.887Z
SHA-256 of the archived text
8a74a01957473674043c3dee585ea5d22a49bd7cd16847284d2f208beb10c05b
Stable URL of this page
https://metraipolis.com/documento/dc_4783e8968809

How to cite

OpenFrontiers: A Collection of Open Research Problems and a Benchmark for Scientific Rediscovery — arXiv:2610.37739 — arXiv (en), https://arxiv.org/abs/2610.37739 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4783e8968809 (Metraipolis archive dc_4783e8968809, sha256 8a74a01957473674043c3dee585ea5d22a49bd7cd16847284d2f208beb10c05b)
BibTeX
@misc{dc4783e8968809,
  title = {OpenFrontiers: A Collection of Open Research Problems and a Benchmark for Scientific Rediscovery — arXiv:2610.37739},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.37739},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4783e8968809; sha256 8a74a01957473674043c3dee585ea5d22a49bd7cd16847284d2f208beb10c05b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.37739. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) have shown increasing promise in research-level problem solving, yet a comprehensive collection of such problems is still missing, and evaluating them on research-level problems remains challenging. We introduce OpenFrontiers, a cross-disciplinary resource that combines a structured corpus of open research problems with a retrospective benchmark of recently resolved problems in mathematics. The corpus records problem statements, context, progress, and other information for 12,151 problems. The benchmark contains 200 problems, 110 resolved in 2026 and 90 still open, each with a Lean~4 statement checked by automated probes and two independent reviews; a subset HQ-50 further filters based on quality score, etc. Our evaluation protocol grades formal proofs with two independent checkers in an isolated sandbox under closed-book, frozen-retrieval, and live-web conditions. Evaluating seven LLM agents on the HQ-50 subset under a standardized budget, we find that closed-book rediscovery is rare, at most 7 of 40 resolved problems, whereas live-web success mostly reflects retrieval of published solutions. By linking a broad research-problem corpus to a formally graded evaluation subset with documented solutions, OpenFrontiers provides a foundation for measuring research-level rediscovery and studying progress toward solving currently open scientific problems. The dataset and benchmark could be found in this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.37739 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T06:07:08.422Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).