Metraipolis › archive › dc_c2a92dc287fe
DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents — arXiv:2609.02059
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.02059
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T03:25:49.618Z
- SHA-256 of the archived text
b5a397281109a5aff7263953d00ba43e957267e97e68a2d4dad222c75600aa44- Stable URL of this page
https://metraipolis.com/documento/dc_c2a92dc287fe
How to cite
DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents — arXiv:2609.02059 — arXiv (en), https://arxiv.org/abs/2609.02059 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c2a92dc287fe (Metraipolis archive dc_c2a92dc287fe, sha256 b5a397281109a5aff7263953d00ba43e957267e97e68a2d4dad222c75600aa44)
BibTeX
@misc{dcc2a92dc287fe,
title = {DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents — arXiv:2609.02059},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.02059},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c2a92dc287fe; sha256 b5a397281109a5aff7263953d00ba43e957267e97e68a2d4dad222c75600aa44},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02059. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multimodal Large Language Models (MLLMs) have achieved strong performance on structured visual understanding tasks such as chart and document question answering. However, existing benchmarks typically evaluate these domains in isolation, leaving underexplored a key capability: whether models can use textual context to determine how chart evidence should be selected, interpreted, and aggregated. We introduce DocHop, a benchmark for integrated chart--context reasoning in document-style images. In DocHop, the document narrative specifies multi-step compositional constraints, while charts provide the corresponding data values. Questions are grounded on a semantic reference label defined in the narrative, requiring models to resolve target entities from context before aggregating evidence across multiple charts. To enable systematic evaluation, we construct DocHop via a stochastic logic-first generation pipeline with controllable reasoning depth and visual density, covering 2,074 examples across six task categories. Experiments on a wide range of proprietary and open-source MLLMs show a substantial gap to human performance: annotators achieve over 90% accuracy, while the best model reaches only 62.83%. Reasoning-enhanced models consistently show improved results, but performance degrades as reasoning complexity increases. Overall, DocHop provides a controlled testbed for challenging multi-hop document reasoning. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.02059 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T03:13:12.414Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).