Metraipolis › archive › dc_5517795ce99a

Dutch Books for Language Models — arXiv:2609.02797

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.02797
License (as archived): CC0-1.0
Archived: 2026-10-17T03:38:38.904Z
SHA-256 of the archived text: 0da86b0a10c1bcb0f22f072308a5302f76c494b24e03fd7993f1033d5c5422b5
Stable URL of this page: https://metraipolis.com/documento/dc_5517795ce99a

How to cite

Dutch Books for Language Models — arXiv:2609.02797 — arXiv (en), https://arxiv.org/abs/2609.02797 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5517795ce99a (Metraipolis archive dc_5517795ce99a, sha256 0da86b0a10c1bcb0f22f072308a5302f76c494b24e03fd7993f1033d5c5422b5)

BibTeX

@misc{dc5517795ce99a,
  title = {Dutch Books for Language Models — arXiv:2609.02797},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.02797},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5517795ce99a; sha256 0da86b0a10c1bcb0f22f072308a5302f76c494b24e03fd7993f1033d5c5422b5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02797. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

People increasingly use language models to support life decisions. Many such decisions involve a probabilistic forecast: How likely is a major life event, a natural disaster, or an economic outcome? Users of language models may implicitly trust that these forecasts fall out of a coherent world model. In this paper, we evaluate the coherence of language model probabilistic forecasts through a procedure that builds on a theorem due to de Finetti. We elicit forecasts from language models across events generated from stock returns data. We then use linear programs to compute the largest Dutch-book profit - the profit an arbitrageur could guarantee by betting against model-generated probabilities - which we use as a measure of incoherence. Our procedure does not require outcome labels, so we can evaluate coherence even in settings where outcomes are not observed or have not yet resolved. We find substantial evidence of incoherence in language model forecasts. Such incoherence increases when there are richer logical relationships between events, and irrelevant contextual details can increase incoherence by an order of magnitude. We conclude by discussing how alternative training strategies may improve probabilistic coherence.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.AI).
Categorie arXiv: General Economics (econ.GN) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.02797 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T03:13:12.572Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).