Metraipolis › archive › dc_c2a342b90fc8
From Values to Benchmarks: Evaluating Large Language Models for Governmental Use in Dutch — arXiv:2608.09925
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.09925
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:50:33.037Z
- SHA-256 of the archived text
d3bb752c0677c5a2bfefed904660236c418e0d136164a1c606a31a62e96fb4d0- Stable URL of this page
https://metraipolis.com/documento/dc_c2a342b90fc8
How to cite
From Values to Benchmarks: Evaluating Large Language Models for Governmental Use in Dutch — arXiv:2608.09925 — arXiv (en), https://arxiv.org/abs/2608.09925 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c2a342b90fc8 (Metraipolis archive dc_c2a342b90fc8, sha256 d3bb752c0677c5a2bfefed904660236c418e0d136164a1c606a31a62e96fb4d0)
BibTeX
@misc{dcc2a342b90fc8,
title = {From Values to Benchmarks: Evaluating Large Language Models for Governmental Use in Dutch — arXiv:2608.09925},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.09925},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c2a342b90fc8; sha256 d3bb752c0677c5a2bfefed904660236c418e0d136164a1c606a31a62e96fb4d0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.09925. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models are increasingly being deployed in governmental settings, yet few existing evaluation frameworks jointly reflect the values of public administration and the linguistic requirements of non-English contexts. We present the "Grip on LLMs" framework, a systematic evaluation suite for Dutch governmental use developed in collaboration with domain experts from a major Dutch municipal organisation. Through an advisory board process, user research, and a survey of the users of a civil-servant chatbot, we identify six evaluation dimensions (factuality, honesty, social bias, energy consumption, cost, and training data transparency) and operationalise them into a benchmark suite covering more than 30 multilingual and Dutch-specific models. Our results reveal that no single model excels across all dimensions, and that trade-offs are unavoidable: higher quality consistently comes at greater environmental impact and financial cost, while bias remains largely independent of both. We further find that factuality (whether a model answers correctly) and honesty (whether a model acknowledges what it does not know) are governed by distinct properties, with high factuality not implying high honesty. To make these findings actionable for non-technical audiences, we release a publicly accessible, user-friendly model overview designed for the full range of stakeholders involved in governmental LLM selection, from engineers to policymakers. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.09925 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:49:41.950Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).