Metraipolis › archive › dc_0e67cd8bac82

Reinforcement Learning Done Right for Explicit Confidence Targets — arXiv:2610.35059

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.35059
License (as archived)
CC0-1.0
Archived
2026-10-27T08:13:18.618Z
SHA-256 of the archived text
4b08af683aca101b09e80671eedac8feb69654728e0614c5104c570f5b7d6e98
Stable URL of this page
https://metraipolis.com/documento/dc_0e67cd8bac82

How to cite

Reinforcement Learning Done Right for Explicit Confidence Targets — arXiv:2610.35059 — arXiv (en), https://arxiv.org/abs/2610.35059 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0e67cd8bac82 (Metraipolis archive dc_0e67cd8bac82, sha256 4b08af683aca101b09e80671eedac8feb69654728e0614c5104c570f5b7d6e98)
BibTeX
@misc{dc0e67cd8bac82,
  title = {Reinforcement Learning Done Right for Explicit Confidence Targets — arXiv:2610.35059},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.35059},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0e67cd8bac82; sha256 4b08af683aca101b09e80671eedac8feb69654728e0614c5104c570f5b7d6e98},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.35059. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) can produce plausible but incorrect answers when they are uncertain, which can mislead users and undermine model reliability. Training models to abstain with reinforcement learning is a natural approach, but most existing methods use a fixed error penalty and therefore cannot adapt their behavior to different confidence requirements at inference time. We study \emph{explicit confidence targets}, where a user specifies a target \(p\) in the prompt and the model directly chooses whether to answer or abstain, with the goal of achieving an accuracy of at least \(p\) among answered questions. However, learning to respond to different confidence targets with a single model has proven difficult, and prior attempts have failed to distinguish different targets. We show that this failure comes from two problems: the reward signal that distinguishes different values of $p$ is often lost during optimization, while the original error penalty grows without bound as $p$ approaches one. We propose RLEC (\textbf{R}einforcement \textbf{L}earning with \textbf{E}xplicit \textbf{C}onfidence Targets), which preserves the learning signal needed to distinguish confidence targets, introduces a stable reward family that keeps the desired answer condition unchanged, and uses reward scheduling to balance effective learning and training stability. Together, these designs enable the first successful training of a single LLM for explicit confidence target control. Extensive experiments across diverse benchmarks and model scales show that RLEC largely preserves general capabilities and achieves competitive calibration performance. RLEC also supports fine-grained target control without an external decision system or threshold calibration after training. By making explicit confidence targets part of the trained policy, RLEC opens a new and more direct path toward behavioral calibration.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.35059 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:56:00.315Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).