Metraipolis › archive › dc_0da14f2f137d
Accounting for Bias Enables Sustainable LLM Evaluation — arXiv:2609.31184
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.31184
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T01:38:37.176Z
- SHA-256 of the archived text
b492c689b9370c53bc39d17aabbbec5e6aed383ab402452cbaf059bab6110544- Stable URL of this page
https://metraipolis.com/documento/dc_0da14f2f137d
How to cite
Accounting for Bias Enables Sustainable LLM Evaluation — arXiv:2609.31184 — arXiv (en), https://arxiv.org/abs/2609.31184 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0da14f2f137d (Metraipolis archive dc_0da14f2f137d, sha256 b492c689b9370c53bc39d17aabbbec5e6aed383ab402452cbaf059bab6110544)
BibTeX
@misc{dc0da14f2f137d,
title = {Accounting for Bias Enables Sustainable LLM Evaluation — arXiv:2609.31184},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.31184},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0da14f2f137d; sha256 b492c689b9370c53bc39d17aabbbec5e6aed383ab402452cbaf059bab6110544},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31184. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LLM-as-a-judge has become the de facto standard for scalable, subjective evaluation, yet current leaderboards compensate for systematic measurement bias by running ever more comparisons, an approach that is both statistically unsound and computationally wasteful. The root cause is an incomplete measurement model, treating LLM judges as neutral, interchangeable instruments ignores documented biases like position bias, verbosity bias, judge severity, and self-enhancement, that no volume of additional data can eliminate. We propose a unified latent variable framework that jointly models pairwise and ordinal data while explicitly correcting for these confounders, recovering reliable rankings from substantially fewer comparisons. Because fitting this model costs negligible compute relative to a single round of LLM inference, bias correction is not only more statistically rigorous but also a more sustainable approach to trustworthy evaluation. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG); Applications (stat.AP); Methodology (stat.ME). Pagina del lavoro: https://arxiv.org/abs/2609.31184 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T01:12:14.355Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).