Metraipolis › archive › dc_f3da33b555f3

The Probabilistic Structure of Large Language Models — arXiv:2609.25134

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.25134
License (as archived)
CC0-1.0
Archived
2026-10-08T13:46:06.795Z
SHA-256 of the archived text
1d6cee2e0b9ebdfad1f392a8eff8e9f51a8450b127921485a60484791c9da147
Stable URL of this page
https://metraipolis.com/documento/dc_f3da33b555f3

How to cite

The Probabilistic Structure of Large Language Models — arXiv:2609.25134 — arXiv (en), https://arxiv.org/abs/2609.25134 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f3da33b555f3 (Metraipolis archive dc_f3da33b555f3, sha256 1d6cee2e0b9ebdfad1f392a8eff8e9f51a8450b127921485a60484791c9da147)
BibTeX
@misc{dcf3da33b555f3,
  title = {The Probabilistic Structure of Large Language Models — arXiv:2609.25134},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.25134},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f3da33b555f3; sha256 1d6cee2e0b9ebdfad1f392a8eff8e9f51a8450b127921485a60484791c9da147},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25134. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

This paper presents a probabilistic perspective on large language models (LLMs), developed with the aim of bringing together, in a single self-contained account, tools that are usually treated separately across the literature. LLMs are described through probability measures on the set of sequences of tokens, specified via their autoregressive conditional distributions. Training is formulated as a maximum-likelihood estimation problem, addressed by stochastic gradient methods, while text generation is viewed as the sequential simulation of the resulting stochastic process. The role of the asymmetry of the Kullback--Leibler divergence in text generation is examined in relation with characteristic phenomena such as hallucination and the distinction between statistical plausibility and truth. As a complementary illustration of the same viewpoint, we also discuss diffusion models, built around the score function, which cast generation not as sequential token prediction but as the simulation of a reverse-time stochastic process transforming noise into data both in discrete and continuous time.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Probability (math.PR); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.25134 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T13:18:38.203Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).