Metraipolis › archive › dc_b9d0174ff95d

OmniHandwritingOCR: A Diagnostic Benchmark for Evaluating Multimodal LLMs in Handwritten OCR Scenarios — arXiv:2608.18586

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.18586
License (as archived)
CC0-1.0
Archived
2026-10-18T13:45:13.694Z
SHA-256 of the archived text
9e8e968500b2c03ef0aca1208bd2c140ed2cedd11ac3cf26e0d7d26624aefb07
Stable URL of this page
https://metraipolis.com/documento/dc_b9d0174ff95d

How to cite

OmniHandwritingOCR: A Diagnostic Benchmark for Evaluating Multimodal LLMs in Handwritten OCR Scenarios — arXiv:2608.18586 — arXiv (en), https://arxiv.org/abs/2608.18586 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b9d0174ff95d (Metraipolis archive dc_b9d0174ff95d, sha256 9e8e968500b2c03ef0aca1208bd2c140ed2cedd11ac3cf26e0d7d26624aefb07)
BibTeX
@misc{dcb9d0174ff95d,
  title = {OmniHandwritingOCR: A Diagnostic Benchmark for Evaluating Multimodal LLMs in Handwritten OCR Scenarios — arXiv:2608.18586},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.18586},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b9d0174ff95d; sha256 9e8e968500b2c03ef0aca1208bd2c140ed2cedd11ac3cf26e0d7d26624aefb07},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.18586. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multimodal large language models (MLLMs) are increasingly used as OCR systems in document and knowledge-processing pipelines, but their ability to faithfully read real handwriting remains underexplored. Existing OCR benchmarks focus largely on printed text or clean single-line inputs, leaving limited coverage of realistic handwritten OCR scenarios such as multilingual handwriting, writer errors, and structurally complex mathematical expressions. We introduce OmniHandwritingOCR, a diagnostic benchmark for evaluating MLLMs and OCR systems on handwritten OCR. It covers handwritten text recognition and handwritten mathematical expression recognition across six subtasks and twelve subsets, totaling 77.57K labeled images from public datasets and newly collected student writings. A key component is a difficulty-stratified multi-line formula corpus designed to test robustness under increasing structural complexity. We evaluate thirteen open- and closed-source systems with five complementary metrics under a unified protocol. Results show that current systems remain far from faithful transcription: performance drops sharply on complex multi-line formulas, model rankings vary across language and formula settings, and several generative models hallucinate plausible but visually unsupported corrections. OmniHandwritingOCR provides a challenging testbed for diagnosing language, content, structural, and visual-grounding failure modes of multimodal models in handwritten OCR scenarios.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.18586 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T13:41:17.959Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).