Metraipolis › archive › dc_b93c8b52ba74
DeepResearch RewardBench: Evaluating Reward Models for Deep Research Report — arXiv:2611.10156
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.10156
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-10T18:40:45.940Z
- SHA-256 of the archived text
4cf5e626774bf7c18b80f458e7a2cb879376f6f485b7e7e446c8b884d7264afe- Stable URL of this page
https://metraipolis.com/documento/dc_b93c8b52ba74
How to cite
DeepResearch RewardBench: Evaluating Reward Models for Deep Research Report — arXiv:2611.10156 — arXiv (en), https://arxiv.org/abs/2611.10156 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b93c8b52ba74 (Metraipolis archive dc_b93c8b52ba74, sha256 4cf5e626774bf7c18b80f458e7a2cb879376f6f485b7e7e446c8b884d7264afe)
BibTeX
@misc{dcb93c8b52ba74,
title = {DeepResearch RewardBench: Evaluating Reward Models for Deep Research Report — arXiv:2611.10156},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.10156},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b93c8b52ba74; sha256 4cf5e626774bf7c18b80f458e7a2cb879376f6f485b7e7e446c8b884d7264afe},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.10156. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Empowered by Large Language Models (LLMs), deep research systems integrate online information into research reports through multi-step web searching and information synthesis. Evaluating these reports presents a significant challenge due to their long context, high information density, and domain specificity, necessitating the use of AI evaluation as a substitute for prohibitively expensive human assessment. However, few studies have examined the effectiveness of existing reward models or LLMs in understanding and evaluating such research reports. To bridge this gap, we introduce DeepResearch RewardBench, the first benchmark dedicated to evaluating reward modeling for research reports. We collect diverse report candidates from multiple agents or tool-augmented LLMs. These candidates are scored using query-specific golden rubrics, and we ensure data quality and difficulty through a combination of multi-round filtering and human verification. Our benchmark comprises preference datasets formatted as pairs, triplets, and quadruplets to test ranking capabilities across varying levels of difficulty. Extensive experiments demonstrate that while current reward models handle general coherence effectively, their performance significantly degrades when evaluating research reports. Generative LLMs acting as judges demonstrate comprehensive superiority in this domain, with comparative evaluation proving more effective than direct scoring methods. We anticipate that this benchmark will serve as a vital testbed for the evaluation and advancement of research agents. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.10156 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-10T18:37:37.479Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).