Metraipolis › archive › dc_70c797bd9daa
Rubric Curriculum RL: Exploiting the Generation-Verification Gap in Non-Verifiable Domains — arXiv:2611.06244
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.06244
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-05T18:17:48.549Z
- SHA-256 of the archived text
34358df4b90f49858e6e153e0078620d1c7d4797b831b1f00cef901487316227- Stable URL of this page
https://metraipolis.com/documento/dc_70c797bd9daa
How to cite
Rubric Curriculum RL: Exploiting the Generation-Verification Gap in Non-Verifiable Domains — arXiv:2611.06244 — arXiv (en), https://arxiv.org/abs/2611.06244 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_70c797bd9daa (Metraipolis archive dc_70c797bd9daa, sha256 34358df4b90f49858e6e153e0078620d1c7d4797b831b1f00cef901487316227)
BibTeX
@misc{dc70c797bd9daa,
title = {Rubric Curriculum RL: Exploiting the Generation-Verification Gap in Non-Verifiable Domains — arXiv:2611.06244},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.06244},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_70c797bd9daa; sha256 34358df4b90f49858e6e153e0078620d1c7d4797b831b1f00cef901487316227},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.06244. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reinforcement learning with verifiable rewards (RLVR) has led to significant gains in math and code generation. Extending these gains to open-ended domains remains challenging: ground-truth verification is unavailable, human annotation is expensive, and learned reward models are prone to reward hacking. The generation-verification gap, where judging good outputs is easier than producing them, offers a potential path to improvement. However, rubric-based judges are often imprecise and stationary, so policies can overfit or reward hack them. We introduce Rubric Curriculum RL (RcRL), which exploits this gap without new data, human annotations, or stronger teacher models. We combine pairwise preferences over rollouts with a curriculum over rubric criteria, yielding a more discriminative signal than absolute scoring and a non-stationary objective. Whereas baselines plateau within a few hundred steps, RcRL preserves output entropy and keeps improving past 1000 steps. On creative writing, RcRL achieves a 70.5% human win rate against the base model, with gains across benchmarks and judges. On HealthBench, RcRL improves over the base model by 14.6% (full set) and 25.4% (consensus subset), outperforming all training baselines, including instance-specific rubrics. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.06244 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-05T18:12:49.050Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).