Metraipolis › archive › dc_5378add80c69
There Is No Free Lunch in RLVR: Every pass@1 Gain Has a pass@k Price — arXiv:2610.39295
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.39295
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T06:13:10.067Z
- SHA-256 of the archived text
f66e39792f0518cd3c8d383565f977e68342473ceb48af5c19b122ef366dd3bf- Stable URL of this page
https://metraipolis.com/documento/dc_5378add80c69
How to cite
There Is No Free Lunch in RLVR: Every pass@1 Gain Has a pass@k Price — arXiv:2610.39295 — arXiv (en), https://arxiv.org/abs/2610.39295 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5378add80c69 (Metraipolis archive dc_5378add80c69, sha256 f66e39792f0518cd3c8d383565f977e68342473ceb48af5c19b122ef366dd3bf)
BibTeX
@misc{dc5378add80c69,
title = {There Is No Free Lunch in RLVR: Every pass@1 Gain Has a pass@k Price — arXiv:2610.39295},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.39295},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5378add80c69; sha256 f66e39792f0518cd3c8d383565f977e68342473ceb48af5c19b122ef366dd3bf},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.39295. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reinforcement learning with verifiable rewards (RLVR) makes a language model solve more problems on the first try and fewer when given many tries: it raises pass@1 while the base model overtakes it at large sampling budget k. Is this a failure of RL to add reasoning, or a transient that more training removes? We settle the question with a theorem. The reversal is a price, and we compute its exact rate. Modeling RLVR as an exponential tilt of a base policy over strategies shared across prompts, we prove that under exclusive specialization every update that raises pass@1 strictly lowers the success of some prompt; a strictly positive price on prompts (Stiemke duality) is the exact condition, and being merely the best strategy on a prompt is provably not enough. When the loss reaches the hardest prompts, it forces a pass@k crossover, and the number of crossings is at most the sign variation of the shift in per-prompt success, a totally positive bound that is tight and that recovers the known single-crossover result as its order-preserving case. The crossover is not a bug to patch: a constrained update traces the exact pass@1 versus pass@k frontier, and removing the crossover entirely costs the entire pass@1 gain. The same tilt identity yields entropy collapse and the entropy-performance law with an identifiable exponent, unifying two literatures under one mechanism. Reinforcement cannot widen the reasoning frontier; it can only move along it, at a price we make measurable by a diagnostic that reads the bound, the order-preserving test, and the price from any evaluation log. There is no free lunch, only a price. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Optimization and Control (math.OC); Machine Learning (stat.ML). Pagina del lavoro: https://arxiv.org/abs/2610.39295 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T06:07:08.706Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).