Metraipolis › archive › dc_31e4339bddee

AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games — arXiv:2608.06362

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.06362
License (as archived)
CC0-1.0
Archived
2026-10-25T20:52:14.865Z
SHA-256 of the archived text
491b52b32365bcecbb5f09fcd8e32c9f71a8ad32c474d43834c22635a2a85017
Stable URL of this page
https://metraipolis.com/documento/dc_31e4339bddee

How to cite

AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games — arXiv:2608.06362 — arXiv (en), https://arxiv.org/abs/2608.06362 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_31e4339bddee (Metraipolis archive dc_31e4339bddee, sha256 491b52b32365bcecbb5f09fcd8e32c9f71a8ad32c474d43834c22635a2a85017)
BibTeX
@misc{dc31e4339bddee,
  title = {AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games — arXiv:2608.06362},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.06362},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_31e4339bddee; sha256 491b52b32365bcecbb5f09fcd8e32c9f71a8ad32c474d43834c22635a2a85017},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.06362. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Deciding which of two agents is stronger means playing games until skill outweighs luck, and every game costs money, model inference, or expert time. Since the number of games needed is unknown, fixed-budget evaluations either keep paying after the result is settled or stop before the agents can be told apart, while naive optional stopping with an ordinary confidence interval invalidates the stated level. We make such an evaluation stop as soon as its evidence suffices, with the guarantee intact. The Action-Informed Value Assessment Tool (AIVAT) reduces variance in imperfect-information games through conditional mean-zero corrections, by a median $54\times$ across 15 LLM agent configurations spanning 71,439 paired Heads-Up No-Limit Hold'em (HUNL) hands, but does not say when to stop. We combine AIVAT with continuously monitored Confidence Sequences (CSs) into anytime-valid AIVAT (AV-AIVAT), whose online value model learns only from past games so that no game scores its own correction. At the nominal 95\% level and a target precision of $\pm1$ Big Blind, raw outcomes need a median $74\times$ as many hands as AIVAT-corrected outcomes to stop under the Asymptotic CS (AsympCS). Exact finite-sample certification uses the Empirical-Bernstein CS (EB-CS), which needs an independently justified bound on corrected payoffs. We establish such a bound structurally for Leduc hold'em and characterize a width floor set by the CS's bet cap and that bound, which governs how much of a variance gain becomes earlier stopping; the descriptive HUNL EB-CS runs show a median $1.37\times$ stopping-time ratio. AV-AIVAT turns variance reduction into efficient, auditable early stopping while separating asymptotic screening from exact certification, so an evaluation can stop the moment its evidence suffices and hand a third party everything needed to recheck the verdict at that very stopping time.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-07 in cs.MA).
Categorie arXiv: Computer Science and Game Theory (cs.GT) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Multiagent Systems (cs.MA).
Pagina del lavoro: https://arxiv.org/abs/2608.06362 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-25T20:47:44.149Z dalla pagina https://arxiv.org/catchup/cs.MA/2026-08-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).