Metraipolis › archive › dc_d921850c28db
Probe with Participation Trophies: Random-Reward RL as a Probe of LLM Capability — arXiv:2610.01066
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.01066
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T13:54:48.560Z
- SHA-256 of the archived text
ec1ca824b95aade8b079544318835d1d6e2d0d284b51e81e5fd8ecafa6d9a6ae- Stable URL of this page
https://metraipolis.com/documento/dc_d921850c28db
How to cite
Probe with Participation Trophies: Random-Reward RL as a Probe of LLM Capability — arXiv:2610.01066 — arXiv (en), https://arxiv.org/abs/2610.01066 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d921850c28db (Metraipolis archive dc_d921850c28db, sha256 ec1ca824b95aade8b079544318835d1d6e2d0d284b51e81e5fd8ecafa6d9a6ae)
BibTeX
@misc{dcd921850c28db,
title = {Probe with Participation Trophies: Random-Reward RL as a Probe of LLM Capability — arXiv:2610.01066},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.01066},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d921850c28db; sha256 ec1ca824b95aade8b079544318835d1d6e2d0d284b51e81e5fd8ecafa6d9a6ae},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01066. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We connect the spurious-reward paradox to a model's reachability and propose random-reward reinforcement learning (RL) as a useful tool for the probing enterprise, addressing a decade-long debate over what probing performance actually reveals about a model. There are two prevailing explanations for the surprising finding that even random rewards can improve the performance of large language models (LLMs): one attributes the gains to particular mechanisms within RL training; the other to data contamination. Our results motivate a different view: spurious-reward RL can probe a model's reachability, or what further training can attain from its current state under specified constraints, beyond what is reflected in its current performance. Two OLMo checkpoints with the same accuracy on synthetic arithmetic (3.5%), for example, reach 8.5% and 55% in their best runs under the same correctness-rewarded RL. Examining OLMo checkpoints across pre-training and mid-training reveals three distinct regimes of training response: early on, RL produces little improvement even when correct answers are rewarded; later in pre-training, rewarding correct answers becomes effective while random rewards remain weak; and, upon entering mid-training, even random rewards can produce large gains. A similar ordering appears in a number-masked supervised fine-tuning (SFT) analysis of these checkpoints, suggesting that the pattern is not specific to a particular RL mechanism. Moreover, RL with random rewards offers a distinctive perspective on what training can make an LLM do, since its reward signal supplies no information about which answers are correct. By asking what training can attain without correctness feedback, it addresses the label-leakage side of a central problem in decodability-based probing: whether a successful probe reveals the model's capabilities or learns the task itself. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.01066 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T12:15:34.674Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).