Metraipolis › archive › dc_0b16e2fea200
Lifetime-valid decisions by continually evolving AI analysts — arXiv:2610.30562
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.30562
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-23T15:14:59.596Z
- SHA-256 of the archived text
515ececc414a6dd531d1aa83a52a77cb6244f47acc3f6b0d5aa70a7fa750ad0b- Stable URL of this page
https://metraipolis.com/documento/dc_0b16e2fea200
How to cite
Lifetime-valid decisions by continually evolving AI analysts — arXiv:2610.30562 — arXiv (en), https://arxiv.org/abs/2610.30562 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0b16e2fea200 (Metraipolis archive dc_0b16e2fea200, sha256 515ececc414a6dd531d1aa83a52a77cb6244f47acc3f6b0d5aa70a7fa750ad0b)
BibTeX
@misc{dc0b16e2fea200,
title = {Lifetime-valid decisions by continually evolving AI analysts — arXiv:2610.30562},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.30562},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0b16e2fea200; sha256 515ececc414a6dd531d1aa83a52a77cb6244f47acc3f6b0d5aa70a7fa750ad0b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.30562. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
AI agents increasingly act as analysts whose decision can often be cast as hypothesis tests. Classical tests control the Type I error rate, the probability of rejecting a true null hypothesis, for fixed procedures. However, AI analysts introduce internal randomness and can evolve continuously over time. As AI analysts evolve, the risk of approving versions with Type I error rates above the target can accumulate, and adaptive use of evaluation feedback can lead to overfitting. We introduce continual validation, which evaluates each proposed AI analyst version before deployment using simulated data under the null hypothesis, with scenarios that can be randomized and expanded over time. We prove that, regardless of how updates are chosen or how many are proposed, the lifetime probability of ever deploying a version whose Type I error rate exceeds the allowed level remains bounded. This guarantees that every subsequent decision controls the Type I error rate at the target level across these scenarios. At the same time, updates with sufficiently low Type I error rates pass validation with high probability, allowing the analyst to continuously improve its performance without sacrificing rigor. In two examples involving evolving AI analysts for bioequivalence studies and product A/B tests, Type I error rates reached 0.098 and 0.183 without continual validation, compared with at most 0.045 and 0.044 with it, while retaining 82% and 96% of the power from optimally selected valid updates. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Methodology (stat.ME) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Machine Learning (stat.ML). Pagina del lavoro: https://arxiv.org/abs/2610.30562 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-23T15:09:08.532Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).