Metraipolis › archive › dc_e91fad656794

RAGFlip: Measuring Query-Level Negative Flips in Retriever Upgrades — arXiv:2610.07266

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.07266
License (as archived)
CC0-1.0
Archived
2026-10-07T12:34:52.725Z
SHA-256 of the archived text
c93d6a8eadebd0c87a8ca1ac7756ee6fc3888bce645ad5546a5e6615d906fac9
Stable URL of this page
https://metraipolis.com/documento/dc_e91fad656794

How to cite

RAGFlip: Measuring Query-Level Negative Flips in Retriever Upgrades — arXiv:2610.07266 — arXiv (en), https://arxiv.org/abs/2610.07266 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e91fad656794 (Metraipolis archive dc_e91fad656794, sha256 c93d6a8eadebd0c87a8ca1ac7756ee6fc3888bce645ad5546a5e6615d906fac9)
BibTeX
@misc{dce91fad656794,
  title = {RAGFlip: Measuring Query-Level Negative Flips in Retriever Upgrades — arXiv:2610.07266},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.07266},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e91fad656794; sha256 c93d6a8eadebd0c87a8ca1ac7756ee6fc3888bce645ad5546a5e6615d906fac9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.07266. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Retriever upgrades are typically evaluated using aggregate metrics, which can hide regressions on queries the previous retriever already served correctly. We study these regressions as negative flips: queries for which BM25 retrieves a judged relevant passage and the replacement does not. We evaluate BGE-large, E5-large-v2, and SPLADE on three BEIR collections: Natural Questions, HotpotQA, and FiQA, across five retrieval depths. All replacements improve overall retrieval coverage. Negative flips occur in every setting and vary substantially by corpus, retriever, and depth. At k=1, 8.6-37.5% of BM25 successes are lost across the evaluated settings. Negative-flip rates are lower in the larger-depth settings, where the BM25-supported cohort is defined separately at each depth. These rates use the any-relevant support label. On HotpotQA at k=10, requiring every positive qrel passage raises the negative-flip rate to 12.3-17.3%. Simple fixed-budget combinations with BM25 reduce these regressions, and a HotpotQA reader experiment provides a limited downstream check in which some retrieval flips are accompanied by answer regressions. These results motivate evaluating retriever updates using query-level compatibility alongside aggregate retrieval quality.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.IR, nuovi lavori del giorno).
Categorie arXiv: Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2610.07266 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:07:32.546Z dalla pagina https://arxiv.org/list/cs.IR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).