Metraipolis › archive › dc_2d433534db25

OCR-MetaReasoning Benchmark: Evaluating the Meta-Reasoning Ability of MLLMs in Text-Rich Image Understanding — arXiv:2608.30678

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.30678
License (as archived)
CC0-1.0
Archived
2026-10-17T08:07:41.427Z
SHA-256 of the archived text
eb38cfd20703066232f5ef64d64bba74a663c24197a401f9067e972c9579efa5
Stable URL of this page
https://metraipolis.com/documento/dc_2d433534db25

How to cite

OCR-MetaReasoning Benchmark: Evaluating the Meta-Reasoning Ability of MLLMs in Text-Rich Image Understanding — arXiv:2608.30678 — arXiv (en), https://arxiv.org/abs/2608.30678 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2d433534db25 (Metraipolis archive dc_2d433534db25, sha256 eb38cfd20703066232f5ef64d64bba74a663c24197a401f9067e972c9579efa5)
BibTeX
@misc{dc2d433534db25,
  title = {OCR-MetaReasoning Benchmark: Evaluating the Meta-Reasoning Ability of MLLMs in Text-Rich Image Understanding — arXiv:2608.30678},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.30678},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2d433534db25; sha256 eb38cfd20703066232f5ef64d64bba74a663c24197a401f9067e972c9579efa5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.30678. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Text-rich image understanding requires multimodal large language models (MLLMs) to organize OCR (Optical Character Recognition)-grounded evidence across words, layout, fields, charts, and visual correspondences. Existing evaluations often conflate extraction with reasoning and rarely test whether models follow the required reasoning direction: applying visible rules, abstracting hidden regularities, or recovering missing premises. We introduce OCR-MetaReasoning, a controlled single-image benchmark that treats deduction, induction, and abduction as distinct directions and separates final-answer correctness from reasoning-process compliance. The benchmark contains 1,500 verified samples in a balanced \(3\times5\) taxonomy crossing three reasoning types with five OCR-object categories, along with reference reasoning steps, automatic answer scoring, the Meta-Reasoning Macro Score (MRMS), and the Reasoning Process Compliance Score (RPCS). Experiments with representative closed-source and open-source MLLMs show that OCR-grounded meta-reasoning remains far from saturated: models struggle with visible-rule application and layout-sensitive inference, while process-compliant rationales can accompany incorrect final answers under exact-match evaluation. The code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.30678 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:47:52.842Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).