Metraipolis › archive › dc_013c95cd00b3
Language models suffer from a curse of ambiguity — arXiv:2608.15448
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.15448
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T18:58:43.493Z
- SHA-256 of the archived text
a6f06002408f4c41437e06a4e83f3010967c09516a365c09a905269a6f02c1d2- Stable URL of this page
https://metraipolis.com/documento/dc_013c95cd00b3
How to cite
Language models suffer from a curse of ambiguity — arXiv:2608.15448 — arXiv (en), https://arxiv.org/abs/2608.15448 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_013c95cd00b3 (Metraipolis archive dc_013c95cd00b3, sha256 a6f06002408f4c41437e06a4e83f3010967c09516a365c09a905269a6f02c1d2)
BibTeX
@misc{dc013c95cd00b3,
title = {Language models suffer from a curse of ambiguity — arXiv:2608.15448},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.15448},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_013c95cd00b3; sha256 a6f06002408f4c41437e06a4e83f3010967c09516a365c09a905269a6f02c1d2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15448. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models increasingly rely on sampling as a driver of their own improvement, making the fidelity of their learned distributions more critical than ever. Yet, not all distributions are equally easy to learn. In this work, we identify a curse of ambiguity: in large language models, and more broadly in all neural networks that produce discrete probability distributions, the more ambiguous a next-token distribution is, the harder it is to learn accurately. Through an extensive theoretical analysis, we trace this curse to architectural and learning roots. More ambiguous distributions require more capacity to be stored, larger embeddings to be represented, more steps to be fitted, and amplify token-sampling noise. We validate these findings on synthetic tasks with controlled ground truth and observe the same signatures in language models trained on real data. Our results provide a new perspective on the statistical capabilities of large language models and a practical framework for when to trust their output distribution. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE). Pagina del lavoro: https://arxiv.org/abs/2608.15448 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T18:06:57.853Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).