Metraipolis › archive › dc_17a2cccea485
Latent Coherence: Do Frontier Language Models' Probability Judgments Obey the Law of Total Probability? — arXiv:2610.31833
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.31833
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-26T19:52:51.509Z
- SHA-256 of the archived text
4997be3218dd7bc2f6c9fc95985bfbcdbcc57e5575a7dfcecb92bcd4c28ddc0d- Stable URL of this page
https://metraipolis.com/documento/dc_17a2cccea485
How to cite
Latent Coherence: Do Frontier Language Models' Probability Judgments Obey the Law of Total Probability? — arXiv:2610.31833 — arXiv (en), https://arxiv.org/abs/2610.31833 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_17a2cccea485 (Metraipolis archive dc_17a2cccea485, sha256 4997be3218dd7bc2f6c9fc95985bfbcdbcc57e5575a7dfcecb92bcd4c28ddc0d)
BibTeX
@misc{dc17a2cccea485,
title = {Latent Coherence: Do Frontier Language Models' Probability Judgments Obey the Law of Total Probability? — arXiv:2610.31833},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.31833},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_17a2cccea485; sha256 4997be3218dd7bc2f6c9fc95985bfbcdbcc57e5575a7dfcecb92bcd4c28ddc0d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31833. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LawZero's Scientist AI trains a Predictor toward the Bayesian posterior, with an objective that penalizes inconsistent probabilities among latent explanations. We ask whether current frontier language models, with no such training, are already consistent in this sense zero-shot: do their probability judgments satisfy the law of total probability, $P(X)=\sum_z P(z)P(X|z)$, when the latent $z$ must be inferred from records in the context? In separate single-turn calls, models are asked for $P(X)$, for $P(z)$ over the latent's states, and for $P(X|z)$; a causal family applies the same test to $P(y|do(x))$ and the back-door adjustment. Every query has an exact ground-truth probability, and a subset of items is asked in three wordings to separate persistent incoherence from rewording noise. Across seven models, GPT-6.1 Sol and Claude Opus 5.5 are the most coherent (mean absolute residual 0.08 and 0.12 log-odds, about 2 percentage points). Claude Haiku 5.5, Gemini 3.8 Flash and DeepSeek V4-Pro show persistent incoherence (0.27-0.35 log-odds) that is mostly item-specific rather than a shared offset. Within every model, larger residuals go with larger errors against the truth (Spearman 0.40-0.62), so the residual, which needs no ground truth, flags unreliable answers. Reasoning length does not predict coherence, across models or within a model across rewordings of the same question. Simple consistency checks pass almost everywhere; the failures are specific to reasoning through a latent variable. Code and data: this https URL Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.31833 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-26T19:45:48.325Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).