Metraipolis › archive › dc_7aceecea6a1d
CONFIRM: Governing Scientific Reporting for Trustworthy Agentic Neuroimaging — arXiv:2611.01449
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.01449
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T18:09:45.675Z
- SHA-256 of the archived text
1f9c042e6386b3f931a195a49a297d6a3113c5d349973d445a10bfd9a0a8480f- Stable URL of this page
https://metraipolis.com/documento/dc_7aceecea6a1d
How to cite
CONFIRM: Governing Scientific Reporting for Trustworthy Agentic Neuroimaging — arXiv:2611.01449 — arXiv (en), https://arxiv.org/abs/2611.01449 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7aceecea6a1d (Metraipolis archive dc_7aceecea6a1d, sha256 1f9c042e6386b3f931a195a49a297d6a3113c5d349973d445a10bfd9a0a8480f)
BibTeX
@misc{dc7aceecea6a1d,
title = {CONFIRM: Governing Scientific Reporting for Trustworthy Agentic Neuroimaging — arXiv:2611.01449},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.01449},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7aceecea6a1d; sha256 1f9c042e6386b3f931a195a49a297d6a3113c5d349973d445a10bfd9a0a8480f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01449. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language model (LLM) agents can autonomously run neuroimaging analyses from research questions to reported findings. Existing agentic systems use tools to check that code runs and reported values match its output, but rely on LLM judgment to decide which results warrant scientific reporting. We show that LLM judges endorse site-confounded findings even when the confound is presented as evidence. We present \textbf{CONFIRM}, a framework that separates running an analysis from deciding which findings to report. CONFIRM encodes each hypothesis as a machine-readable, pre-specified analysis plan that fixes the claim and its pass criteria before computing results. It then filters results through five deterministic gates (search history and multiplicity, confounds, power, analytic stability, and replication) under three configurable reporting standards, from least to most demanding: \textit{Screen}, \textit{Replicate}, and \textit{Full}. Failed claims receive diagnoses that guide follow-up analyses. To evaluate reporting decisions at the claim level, we introduce \textbf{NeuroClaimBench}, comprising 489 executable neuroimaging claims with frozen data, including literature-supported findings and 150 controls designed to violate individual methodological requirements. On NeuroClaimBench, CONFIRM falsely supports at most 3\% of controls, compared with 23--38\% for the baselines, while recovering up to 61\% of literature-supported claims. When auditing an existing agent, CONFIRM reduces false support on controls from 23\% to 0\% while retaining the agent's literature-supported claims. Its diagnostic feedback also produces more supported revisions than Self-Refine with fewer LLM calls. Together, CONFIRM and NeuroClaimBench show that basing reporting decisions on pre-specified analysis plans and deterministic gates is essential for trustworthy autonomous scientific discovery.
Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MA, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Multiagent Systems (cs.MA).
Pagina del lavoro: https://arxiv.org/abs/2611.01449 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:09:37.688Z dalla pagina https://arxiv.org/list/cs.MA/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).