Metraipolis › archive › dc_6880927097dd
Open-Endedness Bench: Measuring Epistemic Process from Agent Records — arXiv:2610.02588
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.02588
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-05T22:49:04.735Z
- SHA-256 of the archived text
51c81386f3d3ebb0a771423a184693e753628dace29ccd40060341624ea55692- Stable URL of this page
https://metraipolis.com/documento/dc_6880927097dd
How to cite
Open-Endedness Bench: Measuring Epistemic Process from Agent Records — arXiv:2610.02588 — arXiv, https://arxiv.org/abs/2610.02588 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6880927097dd (Metraipolis archive dc_6880927097dd, sha256 51c81386f3d3ebb0a771423a184693e753628dace29ccd40060341624ea55692)
BibTeX
@misc{dc6880927097dd,
title = {Open-Endedness Bench: Measuring Epistemic Process from Agent Records — arXiv:2610.02588},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.02588},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6880927097dd; sha256 51c81386f3d3ebb0a771423a184693e753628dace29ccd40060341624ea55692},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02588. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Agents are increasingly given open-ended research tasks: discovering an empirical law from self-designed experiments, improving a heuristic whose optimum nobody knows, or beating a standing record. Their execution logs record every step of this research, yet the runs are still judged by their outcome score. That score alone does not establish whether an agent's claims follow from executed experiments, and a reference answer may be unavailable. We evaluate the agent's epistemic process: how it forms hypotheses, tests them, and revises them in response to evidence. We introduce OEB (Open-Endedness Bench), a benchmark-agnostic methodology that reads only the agent's execution record and never a reference answer or an outcome score. OEB compiles the record into a unified epistemic event graph whose edges connect the propositions the agent states to the executed actions that test them; each node carries an exact excerpt that code verifies against the record. One principle governs scoring: prose can state a proposition, but only evidence returned by an executed action can support or refute it, so OEB checks what the agent writes against what it actually ran. From the graph, OEB scores four competence axes (evidence, experiment, revision, and no reward hacking), mostly as the share of opportunities for sound research that the agent took, and profiles six subjective persona traits that describe the agent's research habits. We score 119 existing runs over 12 tasks from three benchmarks: LLM post-training, chip design, and a training-speed record. Against logged results, only 16-29% of the improvements agents claim are real. On 9 of 10 tasks, the best run tries more new ideas in its second half than the worst run. The persona readings follow the model: for every trait, the model that ran explains more of its variance across runs than the task (a median of 43% against 7%). Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.02588 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-05T22:45:55.604Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).