Metraipolis › archive › dc_d5feb458f2bd

When Confidence Rises Too Early: Detecting Shortcut Reasoning via Premature Answer Commitment — arXiv:2609.35074

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.35074
License (as archived): CC0-1.0
Archived: 2026-10-07T21:53:36.220Z
SHA-256 of the archived text: 6d9479179084c17ab9f00c82051b5b2e5bc692e90b5a54a10857370d6c5df016
Stable URL of this page: https://metraipolis.com/documento/dc_d5feb458f2bd

How to cite

When Confidence Rises Too Early: Detecting Shortcut Reasoning via Premature Answer Commitment — arXiv:2609.35074 — arXiv (en), https://arxiv.org/abs/2609.35074 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d5feb458f2bd (Metraipolis archive dc_d5feb458f2bd, sha256 6d9479179084c17ab9f00c82051b5b2e5bc692e90b5a54a10857370d6c5df016)

BibTeX

@misc{dcd5feb458f2bd,
  title = {When Confidence Rises Too Early: Detecting Shortcut Reasoning via Premature Answer Commitment — arXiv:2609.35074},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35074},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d5feb458f2bd; sha256 6d9479179084c17ab9f00c82051b5b2e5bc692e90b5a54a10857370d6c5df016},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35074. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The reasoning trajectory of a Large Language Model (LLM) is often treated as a verbalized description of its internal reasoning. However, such trajectories can be unfaithful: a model may rely on shortcuts to reach an answer and then post-rationalize the decision with a seemingly coherent chain of thought. Detecting this shortcut reasoning is challenging because existing monitors and verifiers mainly inspect textual traces or final outcomes, rather than how the model's belief in its answer develops during generation. We introduce ConfLens, a framework that tracks the evolution of confidence in the final answer throughout reasoning. Across three shortcut reasoning settings, we observe a common pattern of premature confidence, where shortcut samples become highly confident in the final answer at early reasoning stages. Existing confidence estimation methods, however, show limited generalizability, reliability, or efficiency for detecting this behavior. We therefore propose the Distributional Answer Commitment Score (DACS), a distributional confidence estimator that measures the entropy of the model's probability distribution over answer commitment at each reasoning step. DACS captures how concentrated the model's answer belief is without requiring ground-truth answers or task-specific verifiers. We further convert ConfLens detection results into interpretable signals for reward models to reduce their preference for shortcut reasoning. Experiments on mathematical and code reasoning tasks show that ConfLens with DACS improves shortcut reasoning detection by over 4.3% F1 compared with strong baselines and reduces the mismatch between faithfulness and correctness in reward model preferences.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.35074 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T21:27:08.656Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).