Metraipolis › archive › dc_2048fe542001
How BERT-based Cross-Encoders Model Relevance in Information Retrieval? — arXiv:2610.41425
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.41425
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T08:20:05.749Z
- SHA-256 of the archived text
7e84a2a04f77cd358a3b145324524afdded3cbfd3f1cd659c108acb1afb1f87c- Stable URL of this page
https://metraipolis.com/documento/dc_2048fe542001
How to cite
How BERT-based Cross-Encoders Model Relevance in Information Retrieval? — arXiv:2610.41425 — arXiv (en), https://arxiv.org/abs/2610.41425 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2048fe542001 (Metraipolis archive dc_2048fe542001, sha256 7e84a2a04f77cd358a3b145324524afdded3cbfd3f1cd659c108acb1afb1f87c)
BibTeX
@misc{dc2048fe542001,
title = {How BERT-based Cross-Encoders Model Relevance in Information Retrieval? — arXiv:2610.41425},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.41425},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2048fe542001; sha256 7e84a2a04f77cd358a3b145324524afdded3cbfd3f1cd659c108acb1afb1f87c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41425. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Cross-encoders are the most effective but also the least efficient models in Information Retrieval. Despite their success, the mechanisms underlying their relevance predictions remain poorly understood, particularly how they process to match queries and documents. This lack of interpretability limits their optimization, can lead to suboptimal retrieval, especially out-of-domain, and raises efficiency concerns for over-parameterized models. To address these challenges, this paper investigates two under-explored aspects of cross-encoder behavior. First, to understand the relevance prediction process, we trace the flow of information at the different stages of the process and between different input segments. We study the nature of the signals that compose this flow and that allow cross-encoders to predict the relevance of a passage to a query. Our work particularly focuses on the matching performed by the cross-encoders between queries and documents. Second, we explore how cross-encoders behave with out-of-domain documents and/or queries. Through a series of interpretability experiments, we evaluate three BERT-based cross-encoder models, varying in size and pre-training strategies, across both in-domain and out-of-domain datasets. Overall, this study both consolidates what is known about how cross-encoders model relevance and extends it by exploring the role of matching signals. Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.IR, nuovi lavori del giorno). Categorie arXiv: Information Retrieval (cs.IR). Pagina del lavoro: https://arxiv.org/abs/2610.41425 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T08:06:23.751Z dalla pagina https://arxiv.org/list/cs.IR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).