Metraipolis › archive › dc_102daa6a8fa2

Human-AI-Powered Hypothesis Testing: Cost-Aware Selective AI Scoring and Sequential Human Escalation — arXiv:2609.28859

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.28859
License (as archived)
CC0-1.0
Archived
2026-10-08T04:25:42.954Z
SHA-256 of the archived text
e14545c0faeaf1e73214dea9353e8e2bf69e798962253ae7693d085fefbe5f8d
Stable URL of this page
https://metraipolis.com/documento/dc_102daa6a8fa2

How to cite

Human-AI-Powered Hypothesis Testing: Cost-Aware Selective AI Scoring and Sequential Human Escalation — arXiv:2609.28859 — arXiv (en), https://arxiv.org/abs/2609.28859 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_102daa6a8fa2 (Metraipolis archive dc_102daa6a8fa2, sha256 e14545c0faeaf1e73214dea9353e8e2bf69e798962253ae7693d085fefbe5f8d)
BibTeX
@misc{dc102daa6a8fa2,
  title = {Human-AI-Powered Hypothesis Testing: Cost-Aware Selective AI Scoring and Sequential Human Escalation — arXiv:2609.28859},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.28859},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_102daa6a8fa2; sha256 e14545c0faeaf1e73214dea9353e8e2bf69e798962253ae7693d085fefbe5f8d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.28859. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models are increasingly used as inexpensive judges to evaluate outputs, label data, and assess whether a system meets a desired quality standard. Yet using AI judgments for formal statistical inference is fundamentally different from simply treating them as ground-truth labels: AI evaluations can be biased or noisy, and rigorous hypothesis testing requires explicit control of type-I and type-II errors. We study how to use AI judgments, together with selective human verification, to conduct a valid hypothesis test at minimum cost. We consider a population of items with hidden binary labels. After choosing a fixed pool of items, the decision maker can selectively query AI, send an item directly to a human, escalate an AI-scored item to a human after observing the AI report, or stop once sufficient evidence has accumulated. We derive an information-theoretic lower bound that captures the minimum cost of achieving prescribed testing errors and characterizes the value of AI information and human verification through a report-dependent information frontier. Motivated by this characterization, we develop SCALE, a sequential cost-aware policy that combines selective AI scoring with adaptive human escalation. SCALE is valid at finite sample sizes and matches the lower bound to first order as the target error probabilities vanish. We further extend the framework to an unknown AI-output model using paired AI-human pilot data. Numerically, SCALE approaches Human-only or AI-only testing when one source clearly dominates, while achieving its largest savings when inexpensive AI judgments and selective human verification are both valuable.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Information Theory (cs.IT); Methodology (stat.ME).
Pagina del lavoro: https://arxiv.org/abs/2609.28859 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:57.680Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).