Metraipolis › archive › dc_cf0362256fe0
Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents — arXiv:2609.11318
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.11318
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T05:24:04.991Z
- SHA-256 of the archived text
7905612761ae75dd756b926acd05713b9555d6989be8f86e5f4588d0d95d919a- Stable URL of this page
https://metraipolis.com/documento/dc_cf0362256fe0
How to cite
Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents — arXiv:2609.11318 — arXiv (en), https://arxiv.org/abs/2609.11318 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cf0362256fe0 (Metraipolis archive dc_cf0362256fe0, sha256 7905612761ae75dd756b926acd05713b9555d6989be8f86e5f4588d0d95d919a)
BibTeX
@misc{dccf0362256fe0,
title = {Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents — arXiv:2609.11318},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.11318},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cf0362256fe0; sha256 7905612761ae75dd756b926acd05713b9555d6989be8f86e5f4588d0d95d919a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11318. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Deep research agents are increasingly capable of web search, tool use, multimodal evidence analysis, and information synthesis. However, existing benchmarks mainly evaluate medium-horizon exploration and rarely test whether agents can sustain long, dependency-heavy research processes. We introduce Mr. LHDR (Multimodal real-world Long-Horizon Deep Research), a benchmark for evaluating real-world deep research over long, irreducible chains of interdependent evidence across eight categories. Each question is constructed from a hidden Node-Relation graph and requires an average of 12.1 necessary intermediate conclusions with a mean dependency depth of 10.4 before reaching a short, unique, and verifiable answer. Questions incorporate multimodal evidence, including images, maps, PDFs, logos, charts, tables, and video frames, with at least one non-text element that changes the reasoning state. Mr. LHDR evaluates both final answers and the correctness of intermediate conclusions under annotated dependencies. We evaluate general models, deep research systems, and agent frameworks using Overall Accuracy (OA), Strict Accuracy (SA), Checklist Score (CS), and Dependency-Aware Checklist Score (DACS). Results show that even the strongest system achieves only 43.1% OA and 34.3% SA, indicating that final-answer accuracy substantially overestimates complete research success. Removing images reduces DACS by 12.6 points, demonstrating the importance of multimodal evidence, while SA consistently declines as reasoning chains become longer. These findings reveal sustained, dependency-consistent evidence integration, rather than isolated fact retrieval, as a key bottleneck for current deep research agents. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-11 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.11318 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T05:17:52.006Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).