Metraipolis › archive › dc_41b76f3facf6
What Outcome RL Buys: Finishing, Reliability, Rarely Reach — arXiv:2610.34894
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.34894
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T08:11:41.531Z
- SHA-256 of the archived text
472ddf4d18b6eba189dec50a067cfbd7aa63ee3facfab1de15023d0b5dc1637f- Stable URL of this page
https://metraipolis.com/documento/dc_41b76f3facf6
How to cite
What Outcome RL Buys: Finishing, Reliability, Rarely Reach — arXiv:2610.34894 — arXiv (en), https://arxiv.org/abs/2610.34894 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_41b76f3facf6 (Metraipolis archive dc_41b76f3facf6, sha256 472ddf4d18b6eba189dec50a067cfbd7aa63ee3facfab1de15023d0b5dc1637f)
BibTeX
@misc{dc41b76f3facf6,
title = {What Outcome RL Buys: Finishing, Reliability, Rarely Reach — arXiv:2610.34894},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.34894},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_41b76f3facf6; sha256 472ddf4d18b6eba189dec50a067cfbd7aa63ee3facfab1de15023d0b5dc1637f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.34894. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Outcome RL raises pass@1 but not pass@k at large k (Yue et al., 2025). But every pass@1 is scored under a token budget. Every recipe also trains under one. On physics and math problems at fixed token budgets, we compare the pass@1 of 20 released RL models (1.5B to 32B) with that of the thinking models they started from. We ask whether each pass@1 gain is changed behaviour or new knowledge. We divide each change in pass@1 into three parts. Finishing measures answering before the budget runs out. Reliability measures being right once finished. Reach measures problems RL solves that its base answered wrongly in all eight tries, minus problems the base solves that RL answered wrongly in all eight. We find four things. (1) Where the base often hits the token budget before answering, most of the change, gain or loss, is finishing (82% at 4k and 8k). Given more tokens, the base closes most of the finishing gap itself. (2) Where the base has enough tokens, the gain is reliability: the RL model is graded right more often, even on problems both finish. (3) By our definition, RL rarely solves what its base cannot. Reach adds at most 2.1 percentage points on any of our four physics and math sets. We checked 144 physics problems that only the RL model solved. Of these, 3.5% hold up when resampled or read. In a random sample, nine of ten new RL steps a human confirmed also appear in the base's samples. (4) In a small-scale RL run on an 8B model, the gain on new problems is finishing, with reliability flat. Supplying each problem's physical setup was not shown to improve on plain RL. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.34894 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:56:00.244Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).