Metraipolis › archive › dc_3b7aa176dafa

Inter-dimension Dependence for Multi-Dimensional Evaluation of Open-Ended Text — arXiv:2608.23783

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.23783
License (as archived)
CC0-1.0
Archived
2026-10-18T06:17:54.018Z
SHA-256 of the archived text
272cecbce74b39fd9fe053b93f0f7a3ce5ece2e53fcd57a923c4f8e8f9c61b34
Stable URL of this page
https://metraipolis.com/documento/dc_3b7aa176dafa

How to cite

Inter-dimension Dependence for Multi-Dimensional Evaluation of Open-Ended Text — arXiv:2608.23783 — arXiv (en), https://arxiv.org/abs/2608.23783 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3b7aa176dafa (Metraipolis archive dc_3b7aa176dafa, sha256 272cecbce74b39fd9fe053b93f0f7a3ce5ece2e53fcd57a923c4f8e8f9c61b34)
BibTeX
@misc{dc3b7aa176dafa,
  title = {Inter-dimension Dependence for Multi-Dimensional Evaluation of Open-Ended Text — arXiv:2608.23783},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.23783},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3b7aa176dafa; sha256 272cecbce74b39fd9fe053b93f0f7a3ce5ece2e53fcd57a923c4f8e8f9c61b34},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.23783. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLM-as-a-judge methods are widely used for evaluating the quality of generated open-ended text. Such evaluations are generally multi-dimensional, since the error patterns in texts can be different for different dimensions. Therefore, reliable LLM judges should evaluate each target dimension independently. To quantify the extent to which LLM judges depend on non-target dimensions when evaluating a target dimension, i.e., inter-dimension dependence, we propose CorrGap. To measure this, CorrGap uses the difference in correlations between LLM-predicted scores and ground truth scores across different groups of texts. Using CorrGap, we show that inter-dimension dependence is pervasive across LLM judges in open-ended text evaluation tasks. To mitigate inter-dimension dependence, we propose DimCheck, a method that iteratively removes unrelated evidence from COTs generated by LLM judges in a step-wise way. We show that DimCheck mitigates inter-dimension dependence and outperforms strong baselines across three LLMs and four tasks. We also show that smaller trained LLMs can approximate larger LLMs in DimCheck, with much lower inference costs.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-26 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.23783 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T06:17:29.342Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-26?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).