Metraipolis › archive › dc_471d2c85183f

Anytime-Valid LLM Leaderboards via Benchmark-weighted and Block-Factorized e-Processes — arXiv:2609.32248

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32248
License (as archived)
CC0-1.0
Archived
2026-10-07T22:36:20.173Z
SHA-256 of the archived text
77c0267c44614b22363cc23636565f5d77934d3dcb29c670247e2942f929bbac
Stable URL of this page
https://metraipolis.com/documento/dc_471d2c85183f

How to cite

Anytime-Valid LLM Leaderboards via Benchmark-weighted and Block-Factorized e-Processes — arXiv:2609.32248 — arXiv (en), https://arxiv.org/abs/2609.32248 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_471d2c85183f (Metraipolis archive dc_471d2c85183f, sha256 77c0267c44614b22363cc23636565f5d77934d3dcb29c670247e2942f929bbac)
BibTeX
@misc{dc471d2c85183f,
  title = {Anytime-Valid LLM Leaderboards via Benchmark-weighted and Block-Factorized e-Processes — arXiv:2609.32248},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32248},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_471d2c85183f; sha256 77c0267c44614b22363cc23636565f5d77934d3dcb29c670247e2942f929bbac},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32248. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model (LLM) leaderboards compare model capabilities by ranking models according to their mean performance on fixed benchmarks. However, variability in evaluation outcomes across runs may produce unsupported claims of model superiority on the benchmark, a risk compounded by leaderboard updates. In this paper, we propose BB-EDGE (Benchmark-Weighted and Block-Factorized e-processes for Directed Graph Evaluation), a principled framework that represents an LLM leaderboard as a directed graph whose edges certify pairwise mean-performance advantages, with anytime-valid family-wise error rate (FWER) control. Concretely, for each direction, BB-EDGE constructs an empirical-Bernstein e-process by factorizing evidence over protocol-defined blocks and assigning stakes proportional to the corresponding block weights, then applies direct e-Holm across these $e$-processes to certify directional advantages as edges. Theoretically, we characterize weight-proportional linear stakes under heterogeneous benchmark-average nulls and prove anytime FWER control under arbitrary within-block and cross-pair dependence. BB-EDGE further supports anytime-valid Top-$k$ certification and simultaneous rank intervals. Extensive experiments on synthetic data and four real-world benchmarks demonstrate that BB-EDGE maintains anytime FWER control while achieving high efficiency.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.32248 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.621Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).