Metraipolis › archive › dc_315970ff35c2

Q2D-Web: A Large-Scale Benchmark for Retrieval in Agentic RAG Systems — arXiv:2609.08887

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.08887
License (as archived)
CC0-1.0
Archived
2026-10-16T12:58:24.168Z
SHA-256 of the archived text
225838130205708f983560404ce2ca9b0c5382ceae82e4558ca89e6c6089be3e
Stable URL of this page
https://metraipolis.com/documento/dc_315970ff35c2

How to cite

Q2D-Web: A Large-Scale Benchmark for Retrieval in Agentic RAG Systems — arXiv:2609.08887 — arXiv (en), https://arxiv.org/abs/2609.08887 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_315970ff35c2 (Metraipolis archive dc_315970ff35c2, sha256 225838130205708f983560404ce2ca9b0c5382ceae82e4558ca89e6c6089be3e)
BibTeX
@misc{dc315970ff35c2,
  title = {Q2D-Web: A Large-Scale Benchmark for Retrieval in Agentic RAG Systems — arXiv:2609.08887},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.08887},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_315970ff35c2; sha256 225838130205708f983560404ce2ca9b0c5382ceae82e4558ca89e6c6089be3e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.08887. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Evaluating first-stage retrievers in large-scale production RAG requires a benchmark that pairs a large-scale corpus with a large set of agent-reformulated search queries based on real user queries and their conversation threads, and that labels many relevant documents per query. No existing public benchmark evaluates this setting: large-scale collections typically provide only a small number of evaluation queries, whereas benchmarks with many queries generally contain only millions of documents. Moreover, most benchmarks assess human-written queries, while the first-stage retrievers in agentic RAG pipelines serve machine-written reformulations whose distribution differs from human search behavior. To overcome these evaluation gaps, we introduce Q2D-Web (Query2Doc-Web), a large-scale agentic retrieval benchmark consisting of a 190M-document web corpus and 70k agentic search queries in ten languages, reformulated from real-world user queries in production systems. Q2D-Web provides three sets of fixed relevance judgments: agent citations, production rankings, and a combined set that unions both signals and adds LLM-based judgments of unlabeled pooled documents to reduce false negatives. We benchmark 13 retrievers including lexical, dense, and late-interaction models and find that their relative ordering is largely insensitive to the choice of judgment set, while diverging substantially across topical domains, query languages, and query types. To enable fast evaluation, we also study subcorpus sampling as an approximation to full-corpus evaluations. Retaining a third of the corpus, selected by reciprocal rank fusion over pooled retriever runs, preserves the full-corpus model ranking under the combined judgments while raising absolute Recall@1000 only by 4 to 7 points. The public leaderboard is accessible under: this https URL

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CL).
Categorie arXiv: Information Retrieval (cs.IR) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.08887 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T12:30:25.370Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).