Metraipolis › archive › dc_f50cb2cb8944

Can Artificial Intelligence Forecast Treatment Effects in Economics? — arXiv:2610.38883

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.38883
License (as archived)
CC0-1.0
Archived
2026-10-31T06:24:10.322Z
SHA-256 of the archived text
8c98262743f5f847122092c24c196cc24cbddfbaaa37f6363bdd49ac8c1a8811
Stable URL of this page
https://metraipolis.com/documento/dc_f50cb2cb8944

How to cite

Can Artificial Intelligence Forecast Treatment Effects in Economics? — arXiv:2610.38883 — arXiv (en), https://arxiv.org/abs/2610.38883 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f50cb2cb8944 (Metraipolis archive dc_f50cb2cb8944, sha256 8c98262743f5f847122092c24c196cc24cbddfbaaa37f6363bdd49ac8c1a8811)
BibTeX
@misc{dcf50cb2cb8944,
  title = {Can Artificial Intelligence Forecast Treatment Effects in Economics? — arXiv:2610.38883},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.38883},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f50cb2cb8944; sha256 8c98262743f5f847122092c24c196cc24cbddfbaaa37f6363bdd49ac8c1a8811},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.38883. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

I use LLMs to forecast treatment effects of research studies pre-registered on the AEA RCT Registry and the Open Science Framework. I focus on studies whose results were first publicly released in 2026 and elicit forecasts from models whose training or knowledge cut-off dates were before the relevant set of papers was released. First, LLM models extract forecasting questions from the pre-registrations or pre-analysis plans. These questions elicit forecasted treatment effects; not all resolve in the subsequent papers, but those that do can be used to consider forecast accuracy. Forecasts were elicited for 889 treatment effects from 39 papers with results disclosed in January 2026 or later, or 673 from 33 papers after screening. Forecasts were elicited in raw units but I focus on those that could be standardized and therefore compared across papers. Currently, the median absolute forecast error is 0.054 SD for the best model, the average absolute error is 0.093 SD, and the MSE is 0.0218. The AI forecasts perform substantially better than benchmarks forecasting 0 effect or forecasting the mean effect of other papers' results in that time period. Overall, these results suggest considerable scope for forecasts to be of value in informing policy and research.

Tema degli esploratori: economia dei servizi IA (elenco arXiv econ.GN, nuovi lavori del giorno).
Categorie arXiv: General Economics (econ.GN).
Pagina del lavoro: https://arxiv.org/abs/2610.38883 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T06:09:23.299Z dalla pagina https://arxiv.org/list/econ.GN/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).