Metraipolis › archive › dc_7d780a47dc5b
Large Language Models Develop Belief State Geometry In-Context — arXiv:2609.17376
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.17376
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T19:23:34.531Z
- SHA-256 of the archived text
59c251b32dd2a4b67584fcc4a876382e39affa407b7b57255924699643cc1050- Stable URL of this page
https://metraipolis.com/documento/dc_7d780a47dc5b
How to cite
Large Language Models Develop Belief State Geometry In-Context — arXiv:2609.17376 — arXiv (en), https://arxiv.org/abs/2609.17376 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7d780a47dc5b (Metraipolis archive dc_7d780a47dc5b, sha256 59c251b32dd2a4b67584fcc4a876382e39affa407b7b57255924699643cc1050)
BibTeX
@misc{dc7d780a47dc5b,
title = {Large Language Models Develop Belief State Geometry In-Context — arXiv:2609.17376},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.17376},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7d780a47dc5b; sha256 59c251b32dd2a4b67584fcc4a876382e39affa407b7b57255924699643cc1050},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17376. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) trained on next-token prediction exhibit remarkable in-context learning (ICL) abilities, yet the representations that support ICL remain poorly understood. We consider such representations in a controlled setting: prompting LLMs with data emitted from hidden Markov models (HMMs) and probing for the corresponding belief state -- the posterior distribution over the HMM's hidden states given the observed token history. Across six open-source LLMs prompted with data from 40 HMMs selected for non-trivial belief structure, we find that belief states are linearly decodable from residual stream activations, with peak probe $R^2$-values from 0.83-0.99 across HMM and LLM combinations, ranging from early to late layers. To establish functional relevance, we intervene directly on the probe-identified subspace via patching and steering, resulting in downstream prediction quality on the order of the untampered model, while controls degrade performance substantially. Together, these results provide representation-level evidence that ICL in open-source LLMs approximates optimal Bayesian prediction over a context-inferred generative model. More broadly, our findings extend prior results linking input-distribution structure to activation geometry: from toy networks trained explicitly on HMM data to production-scale LLMs. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.CL). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.17376 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T18:32:32.910Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).