Metraipolis › archive › dc_b8f92b2bd504

Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents — arXiv:2609.17708

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.17708
License (as archived)
CC0-1.0
Archived
2026-10-15T13:49:13.614Z
SHA-256 of the archived text
b964624b66fb6e72dc0d03602e3b6b2169f86d1fefefd1387321a2268cd0cf30
Stable URL of this page
https://metraipolis.com/documento/dc_b8f92b2bd504

How to cite

Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents — arXiv:2609.17708 — arXiv (en), https://arxiv.org/abs/2609.17708 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b8f92b2bd504 (Metraipolis archive dc_b8f92b2bd504, sha256 b964624b66fb6e72dc0d03602e3b6b2169f86d1fefefd1387321a2268cd0cf30)
BibTeX
@misc{dcb8f92b2bd504,
  title = {Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents — arXiv:2609.17708},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.17708},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b8f92b2bd504; sha256 b964624b66fb6e72dc0d03602e3b6b2169f86d1fefefd1387321a2268cd0cf30},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17708. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reliable confidence estimation is increasingly central to the trustworthy deployment of language models: a calibrated estimate of the probability that an output is correct decides what to ship, what to escalate, and what to retry. Existing confidence estimators, however, share one design premise: they only read the current inference process, either by introspecting on it, scoring its token probabilities, or resampling it. We argue that the current inference is not a sufficient basis for confidence. We propose XConf (eXperiential Confidence): estimating confidence together with the model's accumulated experience. The experience is stored as a record of the model's own graded past episodes, each holding the task, the model's reflection, its stated confidence, the outcome, and a lesson written once the grade arrived. Given a new task, XConf's Recall stage retrieves past episodes on similar tasks met with a similar stated confidence, and reads off their historical success rate; its Reflect stage shows the model this record, has it name its recurring failure mode, and restate a confidence now informed by its own track records. Our estimator is format-general, requiring no logit access or weight updates, and costs only one answer generation. Across nine benchmarks spanning reasoning, coding, multimodal QA, and interactive agents, and four models from three families, XConf beats or matches ten-sample self-consistency in discrimination (AUROC) on 23 of 24 comparisons, with much lower calibration error (ECE), at a tenth of the generation cost. Used for selective prediction, abstaining on the 10% least-confident episodes raises the delivered success rate by up to 8.7 points on agent tasks. We therefore see experiential confidence estimation as a new paradigm for future general-purpose confidence estimation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.17708 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:48:24.271Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).