Metraipolis › archive › dc_bc96ec5f955d
The Promises and Limitations of Evaluation-Based AI Governance: Evidence from Expert Interviews — arXiv:2611.00465
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00465
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:14:43.599Z
- SHA-256 of the archived text
2692ce75a62bcda981873f04e6a66817617ab558ca286719e8035baf7ba055e0- Stable URL of this page
https://metraipolis.com/documento/dc_bc96ec5f955d
How to cite
The Promises and Limitations of Evaluation-Based AI Governance: Evidence from Expert Interviews — arXiv:2611.00465 — arXiv (en), https://arxiv.org/abs/2611.00465 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bc96ec5f955d (Metraipolis archive dc_bc96ec5f955d, sha256 2692ce75a62bcda981873f04e6a66817617ab558ca286719e8035baf7ba055e0)
BibTeX
@misc{dcbc96ec5f955d,
title = {The Promises and Limitations of Evaluation-Based AI Governance: Evidence from Expert Interviews — arXiv:2611.00465},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00465},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bc96ec5f955d; sha256 2692ce75a62bcda981873f04e6a66817617ab558ca286719e8035baf7ba055e0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00465. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We conducted 37 semi-structured expert interviews to examine how frontier AI evaluations reduce societal-level AI risks. Evaluations have become central to AI risk governance, despite their shortcomings being widely critiqued. Still, the case for how they reduce risks in practice remains largely implicit. Drawing on expert accounts, we reconstruct this case by identifying the mechanisms through which evaluations are expected to reduce risk, the assumptions underpinning those mechanisms, and the challenges those assumptions face. Our findings suggest that `evaluation-based governance' could contain foundational gaps which undermine risk-reduction strategies that depend on evaluations. Using thematic analysis, we identified three key pathways through which evaluations could reduce risk: by (i) making societal-level risks concrete to decision-makers and the public (raised in 18 out of 37 interviews), (ii) serving as a basis for governance commitments and regulation (18), and (iii) incentivizing a better risk culture within AI companies (16). However, these pathways often depended on shared assumptions that participants themselves doubted. The main challenges included, in order of frequency: (i) diminishing informativeness of evaluation results as model capabilities improve (21), (ii) measurement limitations that may be inherent to the evaluation paradigm (18), (iii) a lack of guarantees that evaluation results will be acted upon by companies or governments (16), and (iv) the degree of control and discretion AI companies retain over evaluation design, disclosure, and response (12). Participants also noted that evaluation-based governance could prove counterproductive, by functioning as a permission structure for developers' continued capability improvement (4). We finish by discussing the implications of our findings for evaluation-based governance, and what would improve its prospects. Tema degli esploratori: governance digitale e beni comuni (elenco arXiv cs.CY, nuovi lavori del giorno). Categorie arXiv: Computers and Society (cs.CY). Pagina del lavoro: https://arxiv.org/abs/2611.00465 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T11:00:52.217Z dalla pagina https://arxiv.org/list/cs.CY/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).