Metraipolis › archive › dc_13c3ccbec896

Mubric: Mutation Testing-Guided Rubric Generation for LLM Evaluation — arXiv:2609.37322

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.37322
License (as archived)
CC0-1.0
Archived
2026-10-07T16:23:45.791Z
SHA-256 of the archived text
ed2ce333abd13a968fdcd50632ce56f9b34d8b5b22b68b5075122f8d5e525855
Stable URL of this page
https://metraipolis.com/documento/dc_13c3ccbec896

How to cite

Mubric: Mutation Testing-Guided Rubric Generation for LLM Evaluation — arXiv:2609.37322 — arXiv (en), https://arxiv.org/abs/2609.37322 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_13c3ccbec896 (Metraipolis archive dc_13c3ccbec896, sha256 ed2ce333abd13a968fdcd50632ce56f9b34d8b5b22b68b5075122f8d5e525855)
BibTeX
@misc{dc13c3ccbec896,
  title = {Mubric: Mutation Testing-Guided Rubric Generation for LLM Evaluation — arXiv:2609.37322},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37322},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_13c3ccbec896; sha256 ed2ce333abd13a968fdcd50632ce56f9b34d8b5b22b68b5075122f8d5e525855},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37322. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Rubric-based evaluation is widely used to assess LLM-based systems by decomposing response quality into task-specific scoring criteria. However, automatically generating rubrics that reliably capture task-specific quality requirements remains challenging. We introduce Mubric, a mutation testing-guided approach to rubric generation. Mutation testing, a classic software testing methodology, evaluates a test suite by injecting faults into programs and checking whether the tests detect them. We draw an analogy between test suites and rubrics: if a rubric captures an important quality requirement, introducing a corresponding defect into an otherwise high-quality response should reduce its score. Mubric first mines common defects from real pairs of preferred and dispreferred responses and abstracts these defects into reusable mutation operators, each specifying how to introduce a particular type of response defect. For a new task, it applies relevant operators to a reference response, checks whether the injected defects reduce response quality, and uses insufficiently penalized defects to refine the rubric. We evaluate Mubric on 703 tasks across four representative domains against six advanced rubric generation methods. Mubric achieves the highest overall evaluation accuracy, outperforming the strongest baseline by 7.48 percentage points.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2609.37322 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T15:54:53.615Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).