Metraipolis › archive › dc_b81efaa1d48e
Can Language Models Learn to Reject Their Own Bad Reasoning Steps? — arXiv:2610.05976
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.05976
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:28:27.016Z
- SHA-256 of the archived text
b30be213392c47cf6223add0fc5fc9d32ad14d7c6c26538085e676380e8c2206- Stable URL of this page
https://metraipolis.com/documento/dc_b81efaa1d48e
How to cite
Can Language Models Learn to Reject Their Own Bad Reasoning Steps? — arXiv:2610.05976 — arXiv, https://arxiv.org/abs/2610.05976 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b81efaa1d48e (Metraipolis archive dc_b81efaa1d48e, sha256 b30be213392c47cf6223add0fc5fc9d32ad14d7c6c26538085e676380e8c2206)
BibTeX
@misc{dcb81efaa1d48e,
title = {Can Language Models Learn to Reject Their Own Bad Reasoning Steps? — arXiv:2610.05976},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.05976},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b81efaa1d48e; sha256 b30be213392c47cf6223add0fc5fc9d32ad14d7c6c26538085e676380e8c2206},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05976. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Verifier-guided decoding can prevent harmful reasoning steps from contaminating subsequent generation, but typically relies on an external learned verifier. We ask whether a language model can instead reject its own bad reasoning steps. We define a prefix's recoverability as the probability that the frozen generator can complete it correctly. Diagnostics show that adjacent recoverability changes are often difficult to resolve with practical Monte Carlo budgets, while same-prefix candidates exhibit a sparse low-recoverability tail. We introduce Self-Step Rejection (SSR), which trains a lightweight LoRA acceptance gate on the generator backbone while keeping the base model frozen. SSR uses confidence-qualified first-passage supervision: steps before the first resolved crossing of a root-relative recoverability barrier are accepted, the crossing step is rejected, and unresolved steps and suffixes are excluded. Training combines pointwise classification, same-prefix pairwise learning, and group-relative policy refinement using final-answer correctness. At inference, SSR accepts candidates or resamples from the unchanged prefix under rejection budgets, without an external learned verifier. Across three reasoning models and five mathematical reasoning benchmarks, SSR improves macro-average accuracy over single-pass decoding by 5.4--10.1 points using 1.21--1.40x as many generated tokens, and achieves the highest macro-average accuracy among evaluated step-level methods. Full-solution scaling methods require 4.47--8.27x the single-pass token cost for comparable performance. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.05976 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:20.352Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).