Metraipolis › archive › dc_84a4685dc5ce

Augmented Hypothesis Testing with Persona-Based LLM Simulations — arXiv:2609.24629

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.24629
License (as archived)
CC0-1.0
Archived
2026-10-09T07:07:52.304Z
SHA-256 of the archived text
0eee572af6f93e978f185b36216b3bdc06b74f0544e62a8db0ebc8fb404b3d6e
Stable URL of this page
https://metraipolis.com/documento/dc_84a4685dc5ce

How to cite

Augmented Hypothesis Testing with Persona-Based LLM Simulations — arXiv:2609.24629 — arXiv (en), https://arxiv.org/abs/2609.24629 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_84a4685dc5ce (Metraipolis archive dc_84a4685dc5ce, sha256 0eee572af6f93e978f185b36216b3bdc06b74f0544e62a8db0ebc8fb404b3d6e)
BibTeX
@misc{dc84a4685dc5ce,
  title = {Augmented Hypothesis Testing with Persona-Based LLM Simulations — arXiv:2609.24629},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.24629},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_84a4685dc5ce; sha256 0eee572af6f93e978f185b36216b3bdc06b74f0544e62a8db0ebc8fb404b3d6e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.24629. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A/B testing requires large sample sizes, long timelines, and significant costs. When auxiliary predictions of experimental outcomes are available from machine learning models, uncertain prediction quality precludes replacing human experiments entirely, yet these predictions may still contain useful signal. We propose a principled framework for learning-augmented hypothesis testing that leverages predictions of unknown quality to reduce sample sizes while maintaining statistical validity. Predictions naturally vary in granularity, from coarse aggregate signals to fine-grained individual-level estimates, and our framework addresses both ends of this spectrum: (1) for population-level directional predictions, where only a binary signal on the treatment effect sign is available, we use an asymmetric test and prove consistency and robustness bounds within the learning-augmented algorithms paradigm; (2) for individual-level predictions, we introduce Generalized PPI++ (GPPI), extending Prediction-Powered Inference to handle nonlinear prediction errors through higher-dimensional transformations. Both methods benefit from accurate predictions while remaining robust to inaccurate or adversarial ones. We validate our framework using persona-based LLM simulations, where AI agents equipped with user personas predict individual behavior, as a natural prediction source spanning both granularity levels. Experiments on four real-world datasets demonstrate that our methods, combined with persona-based predictions, substantially reduce experimental costs while preserving rigorous statistical validity.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Applications (stat.AP).
Pagina del lavoro: https://arxiv.org/abs/2609.24629 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:58:09.436Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).