Metraipolis › archive › dc_2204515fe7d2
Learning to Reason with Selective Internal Feedback — arXiv:2610.21392
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.21392
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T09:03:09.565Z
- SHA-256 of the archived text
612f6d56e4a70dad5d84774279f63a54d78fb7721488d1cddf96890890b89b78- Stable URL of this page
https://metraipolis.com/documento/dc_2204515fe7d2
How to cite
Learning to Reason with Selective Internal Feedback — arXiv:2610.21392 — arXiv (en), https://arxiv.org/abs/2610.21392 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2204515fe7d2 (Metraipolis archive dc_2204515fe7d2, sha256 612f6d56e4a70dad5d84774279f63a54d78fb7721488d1cddf96890890b89b78)
BibTeX
@misc{dc2204515fe7d2,
title = {Learning to Reason with Selective Internal Feedback — arXiv:2610.21392},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.21392},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2204515fe7d2; sha256 612f6d56e4a70dad5d84774279f63a54d78fb7721488d1cddf96890890b89b78},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21392. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reinforcement learning from internal feedback can improve language models without ground-truth answers or external verifiers. However, confidence and correctness can diverge during training, and continued optimization of average self-certainty can increase confidence while degrading reasoning accuracy. We introduce SIF, a reinforcement learning method that combines comparative reward construction with selective policy updates. SIF compares ranked token-level self-certainty profiles across multiple windows and peers, assigning positive rewards only when both levels of voting provide sufficient support. Sample and group level filtering determines which responses contribute to policy optimization. Experiments on three backbones across 12 mathematical and scientific reasoning benchmarks show that SIF's final checkpoints achieve higher average accuracy than the best-validation checkpoints of Intuitor and Vigor, with a gain of 4.71 percentage points over Intuitor on Qwen2.5-Math-1.5B. Further analyses show better retention of accuracy during continued optimization, while test-time training experiments demonstrate sustained improvements with as few as four rollouts per prompt. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.21392 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:44:17.735Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).