Metraipolis › archive › dc_3539d4dbe291

Judging the Justification: Decision Certificate Supervision for Generative Reward Models — arXiv:2610.27528

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.27528
License (as archived)
CC0-1.0
Archived
2026-10-21T07:08:56.003Z
SHA-256 of the archived text
6874028db7554f4d8ecf523a0f53ab5399697a9e6d44112e20862269d9a2ad1e
Stable URL of this page
https://metraipolis.com/documento/dc_3539d4dbe291

How to cite

Judging the Justification: Decision Certificate Supervision for Generative Reward Models — arXiv:2610.27528 — arXiv (en), https://arxiv.org/abs/2610.27528 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3539d4dbe291 (Metraipolis archive dc_3539d4dbe291, sha256 6874028db7554f4d8ecf523a0f53ab5399697a9e6d44112e20862269d9a2ad1e)
BibTeX
@misc{dc3539d4dbe291,
  title = {Judging the Justification: Decision Certificate Supervision for Generative Reward Models — arXiv:2610.27528},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.27528},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3539d4dbe291; sha256 6874028db7554f4d8ecf523a0f53ab5399697a9e6d44112e20862269d9a2ad1e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.27528. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Generative reward models (GenRMs) generate natural language rationales to justify preference judgments, yet a correct verdict does not guarantee a sound justification. Existing approaches often rely on reference explanations to supervise rationales, but obtaining such references can be costly and does not necessarily establish whether generated rationales support the resulting verdict. To address these limitations, we propose DeCert, a framework that evaluates the validity and verdict consistency of generated rationales without requiring reference explanations. Crucially, DeCert translates these assessments into preference supervision that accounts for both verdict correctness and justification quality. In particular, when multiple sampled judgments share the same correct verdict, differences in certificate quality enable DeCert to construct rationale preference pairs that distinguish stronger from weaker justifications. Extensive empirical studies across two model families and multiple in- and out-of-distribution benchmarks demonstrate that DeCert consistently outperforms strong generative judge baselines while improving rationale quality. Moreover, DeCert effectively mitigates positional bias, improving position-consistent accuracy by up to 14.8 points. These results highlight the importance of supervising not only which verdict is correct, but also how that verdict is justified.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.27528 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:58.116Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).