Metraipolis › archive › dc_f496a76bd627
All Work And No Play Makes Jack a Dull Boy: Understanding and Preventing Catastrophic Strategy Collapse in RLVR — arXiv:2610.02835
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.02835
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-05T23:09:54.747Z
- SHA-256 of the archived text
ec2253996329ade8a6a8f605a091f0377364e98ae2fcb7cfa4a8e01a943a0916- Stable URL of this page
https://metraipolis.com/documento/dc_f496a76bd627
How to cite
All Work And No Play Makes Jack a Dull Boy: Understanding and Preventing Catastrophic Strategy Collapse in RLVR — arXiv:2610.02835 — arXiv, https://arxiv.org/abs/2610.02835 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f496a76bd627 (Metraipolis archive dc_f496a76bd627, sha256 ec2253996329ade8a6a8f605a091f0377364e98ae2fcb7cfa4a8e01a943a0916)
BibTeX
@misc{dcf496a76bd627,
title = {All Work And No Play Makes Jack a Dull Boy: Understanding and Preventing Catastrophic Strategy Collapse in RLVR — arXiv:2610.02835},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.02835},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f496a76bd627; sha256 ec2253996329ade8a6a8f605a091f0377364e98ae2fcb7cfa4a8e01a943a0916},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02835. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
During post-training of large language models (LLMs) with Reinforcement Learning with Verifiable Rewards (RLVR), GRPO-style algorithms can exhibit severe late-stage collapse. Prompt-based probing reveals that this is not benign strategic pruning, but a harmful contraction of effective strategy capacity that makes distinct reasoning strategies increasingly inaccessible. To characterize this phenomenon, we define strategies through trajectory-level policy-update interactions and develop a unified theoretical framework combining optimization dynamics and information theory. We prove that major RLVR objectives progressively concentrate probability mass onto a single strategy, while sustaining nontrivial task accuracy requires a minimum strategy capacity. The conflict between these two results provides a mechanistic explanation for catastrophic collapse. We further derive the {Mirrored Entanglement Index (MEI)} as a lightweight online warning signal. To prevent collapse, we propose \textbf{Mesh Learning}, which exposes multiple reasoning strategies and prevents any single strategy from dominating optimization. Across AIME26, AIME25, MATH-500, GPQA, and LiveCodeBench, Mesh Learning consistently outperforms strong baselines across Qwen and Phi model families, with gains of up to 13.4 pp and 11.5 pp, respectively. These results establish strategy preservation as a key principle for stable RLVR. Code is available at this https URL .
Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.02835 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T23:01:38.639Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).