Metraipolis › archive › dc_e87d8ddc945f

From Research Goals to Scientific Closure: Evaluating Agents Through Evidence-Supported Physics Research — arXiv:2610.21767

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.21767
License (as archived)
CC0-1.0
Archived
2026-10-16T09:03:58.266Z
SHA-256 of the archived text
67d7eccae8e43b7fe349bd290b621ffd21b81a2a0787c58af9c5094ffd64c994
Stable URL of this page
https://metraipolis.com/documento/dc_e87d8ddc945f

How to cite

From Research Goals to Scientific Closure: Evaluating Agents Through Evidence-Supported Physics Research — arXiv:2610.21767 — arXiv (en), https://arxiv.org/abs/2610.21767 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e87d8ddc945f (Metraipolis archive dc_e87d8ddc945f, sha256 67d7eccae8e43b7fe349bd290b621ffd21b81a2a0787c58af9c5094ffd64c994)
BibTeX
@misc{dce87d8ddc945f,
  title = {From Research Goals to Scientific Closure: Evaluating Agents Through Evidence-Supported Physics Research — arXiv:2610.21767},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.21767},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e87d8ddc945f; sha256 67d7eccae8e43b7fe349bd290b621ffd21b81a2a0787c58af9c5094ffd64c994},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21767. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Can AI agents organize scientific investigations that lead to well-supported conclusions? To evaluate this capability, we introduce PhysClosureBench, a benchmark of 50 expert-verified, goal-specified research tasks spanning strong-field, attosecond, and neighboring physics areas. Each task specifies the research objective and essential scientific constraints, while leaving the agent to organize the investigation and determine how to generate and validate the evidence needed to answer the question. A 20-criterion Atomic Score measures local scientific accomplishment, while Research Closure (RC0--RC4) assesses how fully the investigation resolves the specified research question based on whether its indispensable evidence requirements and applicable scope conditions are satisfied. We evaluate seven frontier LLMs, where Claude-Opus-5 achieves the highest mean Atomic Score (6.29/10) but reaches RC4 in only 32.7% of attempts. Across the benchmark, 36.2% of attempts scoring at least 8/10 still fall short of RC4, while all seven models reach RC4 on substantially fewer tasks in all three runs than in at least one. Case-level analysis further illustrates how misinterpretation of generated results and unresolved physical applicability limits can lead to incomplete closure. Together, these results reveal a persistent gap between AI agents' local scientific accomplishments and their ability to achieve complete research closure.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.21767 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:44:17.788Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).