Metraipolis › archive › dc_1d4314aa86f7

WANDR: A Benchmark for Wide and Deep Research — arXiv:2608.14747

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.14747
License (as archived)
CC0-1.0
Archived
2026-10-18T19:39:52.014Z
SHA-256 of the archived text
20f0a92d449111fc7983135e4f898b31ca325f2b96159656e7b8fdd3e446d9e4
Stable URL of this page
https://metraipolis.com/documento/dc_1d4314aa86f7

How to cite

WANDR: A Benchmark for Wide and Deep Research — arXiv:2608.14747 — arXiv (en), https://arxiv.org/abs/2608.14747 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1d4314aa86f7 (Metraipolis archive dc_1d4314aa86f7, sha256 20f0a92d449111fc7983135e4f898b31ca325f2b96159656e7b8fdd3e446d9e4)
BibTeX
@misc{dc1d4314aa86f7,
  title = {WANDR: A Benchmark for Wide and Deep Research — arXiv:2608.14747},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.14747},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1d4314aa86f7; sha256 20f0a92d449111fc7983135e4f898b31ca325f2b96159656e7b8fdd3e446d9e4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14747. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

WANDR (Wide ANd Deep Research) is a benchmark of 500 realistic, challenging data-collection tasks for research agents. Each task requires a system to discover a large set of entities that satisfy specified criteria (breadth), investigate each entity through multiple coordinated web searches (depth), and return independently verifiable records with supporting sources and excerpts. Tasks are represented as qualification key hierarchies that specify the entities, relationships, evidence, and required count at each level; a hierarchy with n companies, m employees per company, and k sources per employee requires n x m x k records. This structure supports diverse workflows such as market mapping, due diligence, literature review, product comparison, and talent sourcing, with targets ranging from dozens to thousands of records. WANDR replaces static gold answer sets with task-specific judges that refetch cited pages and verify each record against its evidence, allowing evaluation of current and changing facts. Record verdicts are aggregated into soft and hard precision, recall, and F1 scores that distinguish factual quality, coverage, and hierarchical completeness. The tasks are derived from de-identified product-usage logs and produced through a semi-automated pipeline with automated checks, empirical audits, and human review where needed. We evaluate six production research systems and find that the benchmark is far from saturated: at high effort, the strongest system reaches only 0.363 soft F1 and 0.133 hard F1. Performance degrades as target volume and hierarchy depth increase, with incomplete discovery, missing enrichment, and incomplete evidence construction remaining major bottlenecks. The benchmark and evaluation harness are available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.14747 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T19:02:44.050Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).