Metraipolis › archive › dc_d0f4e9a56e92
ChartJudgeBench: Evaluating LMM Judges for Chart-to-Code Generation — arXiv:2609.24210
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.24210
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T22:05:25.563Z
- SHA-256 of the archived text
7ac2b2f7b9baae5302a84d42420ad0bc92c1404437d8c5a624775232e10487a7- Stable URL of this page
https://metraipolis.com/documento/dc_d0f4e9a56e92
How to cite
ChartJudgeBench: Evaluating LMM Judges for Chart-to-Code Generation — arXiv:2609.24210 — arXiv (en), https://arxiv.org/abs/2609.24210 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d0f4e9a56e92 (Metraipolis archive dc_d0f4e9a56e92, sha256 7ac2b2f7b9baae5302a84d42420ad0bc92c1404437d8c5a624775232e10487a7)
BibTeX
@misc{dcd0f4e9a56e92,
title = {ChartJudgeBench: Evaluating LMM Judges for Chart-to-Code Generation — arXiv:2609.24210},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.24210},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d0f4e9a56e92; sha256 7ac2b2f7b9baae5302a84d42420ad0bc92c1404437d8c5a624775232e10487a7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.24210. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Building strong chart-to-code systems increasingly relies on reinforcement learning, whose effectiveness depends critically on the quality of the reward signal. Large Multimodal Models (LMMs) play a natural critical role in jointly assessing chart visual appearance and task requirements. They are therefore increasingly used as visual critics and reward models, yet their reliability as judges remains largely unexplored. To this end, we introduce ChartJudgeBench, a diagnostic vision-language benchmark for assessing LMM judges in chart-to-code workflows. It includes 1,003 Chart Perception Alignment (CPA) instances for pairwise chart comparison and 650 Chart Reasoning Judgment (CRJ) instances for binary Accept/Reject verification in Chart Reproduction and Chart Editing. Together, these tasks emulate the core judging decisions required in agentic refinement and RL-based chart optimization. Our evaluation of strong LMMs reveals four systematic limitations: (i) positional bias in pairwise comparison, (ii) a strong tendency to overpredict Accept, (iii) difficulty in matching visual styles and aesthetics, and (iv) an unexpected leniency bias in RL-trained models. These findings show that current LMM judges require explicit reliability validation before being used as critics or reward models in chart-to-code optimization. The code and data are available on ChartJudgeBench. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.24210 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T21:45:21.009Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).