Metraipolis › archive › dc_c8a5583e9d0c
WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents — arXiv:2609.27490
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.27490
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T09:11:24.787Z
- SHA-256 of the archived text
5dee0e444fd2993954d412a6a8dc72d9839bb6d1bc7efa59371bd748ff75b6a1- Stable URL of this page
https://metraipolis.com/documento/dc_c8a5583e9d0c
How to cite
WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents — arXiv:2609.27490 — arXiv (en), https://arxiv.org/abs/2609.27490 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c8a5583e9d0c (Metraipolis archive dc_c8a5583e9d0c, sha256 5dee0e444fd2993954d412a6a8dc72d9839bb6d1bc7efa59371bd748ff75b6a1)
BibTeX
@misc{dcc8a5583e9d0c,
title = {WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents — arXiv:2609.27490},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.27490},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c8a5583e9d0c; sha256 5dee0e444fd2993954d412a6a8dc72d9839bb6d1bc7efa59371bd748ff75b6a1},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.27490. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
AI research agents must predict the effects of computational changes after budgeted experiments. WhatWorkedBench evaluates this experimental understanding through a delivered response surface of configuration scores. Agents inspect workflow code and buy measurements; exhaustive CPU references score conditional component effects, mean pair interactions, configuration choice, and delivery. The catalog spans 36 task conditions, 30 sources, 8 workflow families, and 1248 indexed configuration records, with 4,206 numerical controls. At eight purchased measurements plus two free anchors, pair-effect ridge selects an exact optimum on 15 of 22 sources; 13 of these cases have at least one conditional-effect error exceeding 10% of the task utility range. Shared-estimator comparisons measure acquisition and reconstruction on common observations. In a prospective typed study on 12 four-factor sources, DeepSeek Flash submits 12/12 direct tables and gains 0.147 recovery over a Gaussian process (GP) fitted to the same observations. Pro delivers 11/12 artifacts, with an all-attempt GP difference of -0.001 and a delivered-only difference of +0.063. On six prespecified new agent-evaluation sources, Flash and Pro gains are 0.149 and 0.074. In eight typed six-factor episodes, seven final tables satisfy verified code equivalences. Four fresh agents pass all six registered rules through named estimators and deliver consistent tables at 0.682 recovery versus 0.710 for separate direct-table runs. WhatWorkedBench links acquisition, inference, program structure, and delivery. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.27490 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T08:13:26.514Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).