Metraipolis › archive › dc_e8de8f6ef96d
A Systematic Multi-Domain Evaluation of Document Retrievers — arXiv:2609.29455
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.29455
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T06:43:56.403Z
- SHA-256 of the archived text
5ebfa6a00f897207d1e97d23174c00f4c3c88ca4f4a230b73812bca0c80a0def- Stable URL of this page
https://metraipolis.com/documento/dc_e8de8f6ef96d
How to cite
A Systematic Multi-Domain Evaluation of Document Retrievers — arXiv:2609.29455 — arXiv (en), https://arxiv.org/abs/2609.29455 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e8de8f6ef96d (Metraipolis archive dc_e8de8f6ef96d, sha256 5ebfa6a00f897207d1e97d23174c00f4c3c88ca4f4a230b73812bca0c80a0def)
BibTeX
@misc{dce8de8f6ef96d,
title = {A Systematic Multi-Domain Evaluation of Document Retrievers — arXiv:2609.29455},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.29455},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e8de8f6ef96d; sha256 5ebfa6a00f897207d1e97d23174c00f4c3c88ca4f4a230b73812bca0c80a0def},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29455. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Document retrieval is a crucial component of many modern AI systems, directly influencing their effectiveness, robustness, and fairness in downstream tasks. While recent years have seen a growing number of retrievers, comparative studies in the literature are typically limited in scope or focused on singular benchmarks, domains, or model families. This fragmentation makes it difficult to draw reliable conclusions about the relative strengths, weaknesses, and trade-offs of document retrievers. To address this gap, we conduct a large-scale empirical evaluation of document retrievers, covering three families (sparse, dense, and expansion-based) and evaluating 33 retrievers across seven IR datasets, analyzing retrieval quality, runtime, and failure points. Rather than tuning each model individually, we evaluate every retriever off the shelf, under the configuration reconstructable from its public documentation and a uniform compute budget. Our results show that NV-Embed-v2 achieves the strongest performance on four of the seven datasets, albeit at the cost of substantial query latencies. Among sparse retrievers, we find that SPLADE-v3 rivals the top-performing approach despite much lower latency, and even achieves top scores on MS MARCO. On instruction-following datasets, GritLM delivers the best performance. Finally, an analysis of the retrievers' failure points reveals contrasts between models and families that indicate potential for unrealized gains in retrieval performance. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-25 in cs.IR). Categorie arXiv: Information Retrieval (cs.IR). Pagina del lavoro: https://arxiv.org/abs/2609.29455 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T06:09:07.180Z dalla pagina https://arxiv.org/catchup/cs.IR/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).