Metraipolis › archive › dc_f034a5e4915c
ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals — arXiv:2609.16816
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.16816
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T19:23:27.162Z
- SHA-256 of the archived text
8289ffdf6995e36ec82bf8a343517e0ed3508b6d7adb0e278432ac72e6e99007- Stable URL of this page
https://metraipolis.com/documento/dc_f034a5e4915c
How to cite
ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals — arXiv:2609.16816 — arXiv (en), https://arxiv.org/abs/2609.16816 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f034a5e4915c (Metraipolis archive dc_f034a5e4915c, sha256 8289ffdf6995e36ec82bf8a343517e0ed3508b6d7adb0e278432ac72e6e99007)
BibTeX
@misc{dcf034a5e4915c,
title = {ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals — arXiv:2609.16816},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.16816},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f034a5e4915c; sha256 8289ffdf6995e36ec82bf8a343517e0ed3508b6d7adb0e278432ac72e6e99007},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.16816. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Language model-generated rubrics are increasingly used as reward signals for rubric-based reinforcement learning, LLM-as-a-judge evaluation, and automated grading. Such rubrics are reliable only if they reward honest answers over adversarial answers optimized to exploit them. Yet their robustness to such optimization remains poorly understood. We isolate the hardest regime: impossible tasks, where the prompt pressures the model toward an unsupported conclusion, so the only honest response is to acknowledge the impossibility. We introduce ImpossibleRubrics, a benchmark of 169 impossible tasks spanning six impossibility categories, each paired with a verifiable oracle certificate specifying what an honest answer may and may not claim, together with 48 answerable controls. Rather than providing fixed rubrics, ImpossibleRubrics provides task environments and certificates, allowing rubrics to be generated downstream and then adversarially tested for whether they reward certificate-violating answers. Eleven generators are exploited 8--26% of the time on the unbiased 150-of-169 environment cut; on a deliberately selected stress cut the strongest generator we measured is still exploited 36% while a certificate-faithful rubric is exploited 0%, so what we measure is a rubric-quality gap, not task impossibility. One result runs against intuition. A single generic rubric ("be decisive, penalize hedging") used unchanged for every task is exploited 64% of the time, and seven of the eleven generators are exploited more often than that while writing a rubric tailored to each one. The tailored criteria appear to tell an attacker which claim to fabricate. The problem is not that rubrics are vague; it is that they are specific about the wrong things.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.16816 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T18:32:32.905Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).