Metraipolis › archive › dc_4473159b0d5e

Predictive Credit: Measuring What Scientific Explanations Add to Experimental Forecasts — arXiv:2610.00314

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.00314
License (as archived)
CC0-1.0
Archived
2026-10-06T12:01:47.153Z
SHA-256 of the archived text
d89025cfd39e01a4c89837a79c19ca8cba20360046b181c56bf3ddc3336f524f
Stable URL of this page
https://metraipolis.com/documento/dc_4473159b0d5e

How to cite

Predictive Credit: Measuring What Scientific Explanations Add to Experimental Forecasts — arXiv:2610.00314 — arXiv (en), https://arxiv.org/abs/2610.00314 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4473159b0d5e (Metraipolis archive dc_4473159b0d5e, sha256 d89025cfd39e01a4c89837a79c19ca8cba20360046b181c56bf3ddc3336f524f)
BibTeX
@misc{dc4473159b0d5e,
  title = {Predictive Credit: Measuring What Scientific Explanations Add to Experimental Forecasts — arXiv:2610.00314},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.00314},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4473159b0d5e; sha256 d89025cfd39e01a4c89837a79c19ca8cba20360046b181c56bf3ddc3336f524f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.00314. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Research agents explain planned experiments. We measure predictive credit with paired forecasts sharing an intervention, forecaster, and outcome while varying description, matched explanation, and donor context. Five checks track commitment, delivery, predictive gain, alignment, and known-signal uptake. Across 336 prospective states in controlled learning, 12 Tox21 endpoints, and 24 OpenML tasks, v5's frozen credit decision was inconclusive. Tox21's preregistered ROC AUC interval-score harm test was unmet ($D-M=-.0026$, 95 percent interval [$-.0174$, .0104]); OpenML's joint formation, point-equivalence, and repeatability rule was unmet. Matched point-accuracy gains over description remained unconfirmed, and Tox21/OpenML seed-donor intervals spanned zero. Under requested DeepSeek V4 Pro, matched and donor cards reduced secondary Tox21 drift by 64.5 and 59.1 percent. A DeepSeek V4 Flash replay raised matched point MAE from .01823 to .02020 and missed matched-donor interval-score equivalence. OpenML full-card assignment widened nominal 80 percent intervals by 21 percent, with 49.3 percent coverage versus 51.4 percent for description and content in 66/144 cards. Direct-text Flash delivered all 144 notes without detectable matched point-accuracy gain. A researcher-authored mechanism positive control lowered point MAE by 2.60 percentage points versus description. The protocol measures predictive credit for research-agent benchmarks and scientific forecasting; natural-explanation credit remained unconfirmed at the tested donor resolutions.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.00314 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T10:21:27.855Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).