Metraipolis › archive › dc_17abea970999

Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams — arXiv:2608.12262

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.12262
License (as archived)
CC0-1.0
Archived
2026-10-19T05:16:36.019Z
SHA-256 of the archived text
12550203ae6b72fe9263cf9e7bd23d5083f8edfaa73da2c8ec809e29b7559e61
Stable URL of this page
https://metraipolis.com/documento/dc_17abea970999

How to cite

Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams — arXiv:2608.12262 — arXiv (en), https://arxiv.org/abs/2608.12262 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_17abea970999 (Metraipolis archive dc_17abea970999, sha256 12550203ae6b72fe9263cf9e7bd23d5083f8edfaa73da2c8ec809e29b7559e61)
BibTeX
@misc{dc17abea970999,
  title = {Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams — arXiv:2608.12262},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.12262},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_17abea970999; sha256 12550203ae6b72fe9263cf9e7bd23d5083f8edfaa73da2c8ec809e29b7559e61},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.12262. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multimodal Large Language Models (MLLMs) have been growing the capability for scientific writing and collaboration. For example, OpenAI Prism is a free workspace for scientific writing and collaboration. One important feature in Prism is turning scientific diagrams directly into LaTeX TikZ code. In this paper, we build a benchmark, Diagram-MMU, a multi-modal benchmark designed to assess MLLMs' ability for scientific diagram parsing and understanding. Diagram-MMU features 3.7k curated diagrams and 18.3k human-validated questions across six domains. It evaluates MLLMs on three tasks common in vibe writing workspaces: diagram-to-code parsing, diagram-to-code editing, and diagram question answering, alongside agentic settings per task. The evaluation of 12 MLLMs reveals that diagram-to-code tasks are more challenging than diagram question answering: models can reason well over diagrams but struggle to parse and edit them, underscoring the need for methods to enhance MLLMs' capability in diagram-to-code generation. Under agentic settings, most models improve parsing and editing performance but degrade on question answering, while Claude-4.6 Opus consistently improves across all three tasks. Project Page: this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.12262 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:08:46.782Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).