Metraipolis › archive › dc_d56a3b88166b
Grading Needs a Rubric, Not Intelligence — arXiv:2608.17938
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.17938
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T14:33:47.391Z
- SHA-256 of the archived text
86df59723b68fb32767e7f3129dcfd3bb7a9cb3a1c1492c9ab6653aa6a71b4ae- Stable URL of this page
https://metraipolis.com/documento/dc_d56a3b88166b
How to cite
Grading Needs a Rubric, Not Intelligence — arXiv:2608.17938 — arXiv (en), https://arxiv.org/abs/2608.17938 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d56a3b88166b (Metraipolis archive dc_d56a3b88166b, sha256 86df59723b68fb32767e7f3129dcfd3bb7a9cb3a1c1492c9ab6653aa6a71b4ae)
BibTeX
@misc{dcd56a3b88166b,
title = {Grading Needs a Rubric, Not Intelligence — arXiv:2608.17938},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.17938},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d56a3b88166b; sha256 86df59723b68fb32767e7f3129dcfd3bb7a9cb3a1c1492c9ab6653aa6a71b4ae},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17938. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Small language models can grade open-ended examination answers as reliably as substantially more expensive models when they grade against an explicit rubric. We test this claim as the design principle behind any-to-bench: a frontier model reads source documents once, at ingestion, to extract each question and its rubric; lower-cost models then perform all repeated grading work. We evaluate six cost-efficient model configurations from two model families at three reasoning-effort levels. Each configuration answers 24 open-ended examination questions, and each also grades every answer sheet three times, yielding 3,456 per-question grades. Scores depend overwhelmingly on the answer being graded: answer identity explains 95.6% of score variance, whereas judge identity explains only 0.2%. Raising a writer's reasoning effort moves earned scores by as much as 0.143 of full marks, while raising a judge's reasoning effort moves assigned scores by at most 0.006. Six frontier-tier judges, added as a check, reproduce these scores and are no more reliable as a panel. Two ablations then decompose the rubric on the same questions and answers. Removing its criteria and levels while keeping the official answer changes nothing measurable. Removing the official answer as well collapses reliability (ICC 0.888 to 0.628), inflates scores, and makes judge reasoning effort matter again. The rubric is what decouples grading from judge intelligence, and within the rubric the official answer does nearly all the work. We find no evidence of length preference or same-family preference under rubric-anchored grading. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.17938 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T14:27:54.471Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).