Metraipolis › archive › dc_cbda21d38d9f

Optimal Re-Ranking Depth — arXiv:2611.05908

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.05908
License (as archived)
CC0-1.0
Archived
2026-11-05T18:29:20.250Z
SHA-256 of the archived text
a46ea7aa72026d4a1444e4313b8e180fd4f44260585e08e9d84a7765c5458807
Stable URL of this page
https://metraipolis.com/documento/dc_cbda21d38d9f

How to cite

Optimal Re-Ranking Depth — arXiv:2611.05908 — arXiv (en), https://arxiv.org/abs/2611.05908 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cbda21d38d9f (Metraipolis archive dc_cbda21d38d9f, sha256 a46ea7aa72026d4a1444e4313b8e180fd4f44260585e08e9d84a7765c5458807)
BibTeX
@misc{dccbda21d38d9f,
  title = {Optimal Re-Ranking Depth — arXiv:2611.05908},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05908},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cbda21d38d9f; sha256 a46ea7aa72026d4a1444e4313b8e180fd4f44260585e08e9d84a7765c5458807},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05908. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Second-stage neural rankers are commonly applied with a fixed re-ranking depth, assuming that retrieval effectiveness saturates as depth increases. Prior research has questioned the assumption that increasing the re-ranking depth yields linear performance gains, further suggesting that optimal re-ranking depth varies considerably from query to query. In the past, studying such phenomena was methodologically difficult given the scale of manual annotation required. With the advent of LLM-based relevance judgments we can now more easily undertake such studies, in this case to pinpoint the optimal re-ranking depth on a per-query basis. Using dense LLM-based relevance judgments over a typical re-ranking pipeline, we show that many queries exhibit a well-defined optimal re-ranking depth, beyond which effectiveness stagnates or degrades. We formulate re-ranking depth as a query-specific property and study whether it can be predicted a priori from first-stage retrieval characteristics. Through a large-scale analysis, we find that most standard query performance prediction (QPP) methods are ineffective for this task. In contrast, a predictor derived from LLM-assessed first-stage ranking quality, which we term IR-DCG@10, can reduce the average re-ranking depth by up to a factor of 3 while preserving overall effectiveness, depending on the specific first- and second-stage rankers used. Under oracle selection of optimal depths, we further show that retrieval effectiveness can improve by more than 7% while reducing the average re-ranking depth by a factor of 5 on the MSMARCO DEV collection. Given our promising preliminary findings, we would encourage the use of automatic judgments to facilitate research otherwise infeasible under manual annotation. Towards this point, we release relevance judgments, our codebase, and experimental artefacts to support reproducibility and further research.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.IR, nuovi lavori del giorno).
Categorie arXiv: Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2611.05908 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-05T18:19:33.955Z dalla pagina https://arxiv.org/list/cs.IR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).