Metraipolis › archive › dc_7b38237705ce

Calibrated Enough to Know, Not Calibrated to Act: Fabricated Evidence Makes LLM Agents Commit to the Unknowable — arXiv:2608.27167

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.27167
License (as archived): CC0-1.0
Archived: 2026-10-17T14:23:37.484Z
SHA-256 of the archived text: 58abfa440af09b32d34460fd0e6c5971e1917bfb365bb5f730a409699159693c
Stable URL of this page: https://metraipolis.com/documento/dc_7b38237705ce

How to cite

Calibrated Enough to Know, Not Calibrated to Act: Fabricated Evidence Makes LLM Agents Commit to the Unknowable — arXiv:2608.27167 — arXiv (en), https://arxiv.org/abs/2608.27167 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7b38237705ce (Metraipolis archive dc_7b38237705ce, sha256 58abfa440af09b32d34460fd0e6c5971e1917bfb365bb5f730a409699159693c)

BibTeX

@misc{dc7b38237705ce,
  title = {Calibrated Enough to Know, Not Calibrated to Act: Fabricated Evidence Makes LLM Agents Commit to the Unknowable — arXiv:2608.27167},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.27167},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7b38237705ce; sha256 58abfa440af09b32d34460fd0e6c5971e1917bfb365bb5f730a409699159693c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.27167. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

An LLM agent shown a professional-looking market panel commits to a directional call on a provably unpredictable question far more often than one asked the bare question: across 12 frontier models, commitment rises from 6.5% to 54.0% as evidence is escalated. It commits just as readily when every number on the panel is invented: fabricating the entire display, so nothing the model can see is true except the question itself, still lifts commitment from 24.5% to 36.8%, statistically indistinguishable from the 37.6% produced by genuine market data. What unlocks confident action is not information but the authority of its packaging. The failure is narrow and locatable. Incapacity is not the answer: on matched answerable questions attached to the same panels, the same models answer essentially always, at near-perfect accuracy. Nor is it belief - stated probabilities barely move across the gradient that swings action by 48 points, and score worse than a climatological baseline. Missing judgment isn't it either: asked to classify a question's knowability before acting, models call it irreducible 90% of the time and then commit on just 0.4% of those. The act/don't-act gate is what fails, and the effect is concentrated in a few models rather than universal. Because the gate is separable, it can be trained. Supervised fine-tuning of a 3B model on 540 synthetic cases, predominantly dice, coins, jars and timers, drives commitment to 0.0% on the original cases and transfers to three unseen domains. It does not survive everything: the gate holds exactly when the response format leaves room to reason, and rigid formats that remove that room leave the model confident and wrong on questions it otherwise answers correctly. The gate is trainable and context-fragile, and deployment needs both halves of that sentence.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.27167 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T13:16:53.557Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).