Metraipolis › archive › dc_a5d4060ee8eb

Beyond Verbalized Confidence: Calibrating Reasoners with Differentiable Readouts — arXiv:2609.34857

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.34857
License (as archived)
CC0-1.0
Archived
2026-10-07T21:03:00.463Z
SHA-256 of the archived text
a4c20177e3ea44f24139e04f5c15678904e36ff64bc3405dc08723bde1a5d47b
Stable URL of this page
https://metraipolis.com/documento/dc_a5d4060ee8eb

How to cite

Beyond Verbalized Confidence: Calibrating Reasoners with Differentiable Readouts — arXiv:2609.34857 — arXiv (en), https://arxiv.org/abs/2609.34857 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a5d4060ee8eb (Metraipolis archive dc_a5d4060ee8eb, sha256 a4c20177e3ea44f24139e04f5c15678904e36ff64bc3405dc08723bde1a5d47b)
BibTeX
@misc{dca5d4060ee8eb,
  title = {Beyond Verbalized Confidence: Calibrating Reasoners with Differentiable Readouts — arXiv:2609.34857},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.34857},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a5d4060ee8eb; sha256 a4c20177e3ea44f24139e04f5c15678904e36ff64bc3405dc08723bde1a5d47b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34857. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement learning with verifiable rewards (RLVR) trains reasoning models to produce correct answers, but does not ensure that their stated confidence is calibrated. The resulting models are systematically overconfident. Recent methods train calibration inside the RLVR loop by having the model state a numerical confidence alongside its answer, but they all obtain the confidence by sampling it as text. This choice imposes two costs: a sampled confidence introduces variance and in practice collapses to a handful of distinct values, and sampling makes the confidence non-differentiable, forcing the calibration loss through a scalar reward. We propose CREDO (Confidence REaDOut) to replace sampling with a deterministic readout. While RLVR optimizes correctness, CREDO reads the confidence from a dedicated token pair in the model's output distribution and trains it by differentiable regression. CREDO further turns the trained confidence into a signal for accuracy, weighting rollouts by how far confidence and outcome disagree, so that accuracy and calibration improve together. Across mathematical and code reasoning, CREDO attains the best accuracy and calibration, and the gains extend to abstention and selective prediction.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.34857 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:03.176Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).