Metraipolis › archive › dc_f6b9bf4bc953

Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines — arXiv:2609.24122

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.24122
License (as archived)
CC0-1.0
Archived
2026-10-09T06:50:03.723Z
SHA-256 of the archived text
6520e61f1ff81a6e59163863efc606adea7cdeacc9e5d897295305ad1b7c3ead
Stable URL of this page
https://metraipolis.com/documento/dc_f6b9bf4bc953

How to cite

Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines — arXiv:2609.24122 — arXiv (en), https://arxiv.org/abs/2609.24122 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f6b9bf4bc953 (Metraipolis archive dc_f6b9bf4bc953, sha256 6520e61f1ff81a6e59163863efc606adea7cdeacc9e5d897295305ad1b7c3ead)
BibTeX
@misc{dcf6b9bf4bc953,
  title = {Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines — arXiv:2609.24122},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.24122},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f6b9bf4bc953; sha256 6520e61f1ff81a6e59163863efc606adea7cdeacc9e5d897295305ad1b7c3ead},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.24122. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Retrieval-augmented generation (RAG) is hard to monitor in production: exhaustive relevance labels do not exist for non-stationary multi-million-passage corpora that re-index in real time. As a result, retrieval quality is generally understudied and often deprioritised in favour of generation-oriented metrics. In this work, we propose auditing retrieval coverage by probing for evidence of missing documents rather than enumerating every relevant one. Our method Re:CAP (REtrieval Coverage Audit by iterative Probing) is a reference-free audit loop applied to a deployed RAG pipeline's initial answer and retrieved context: it identifies the topics already covered, generates probing questions for plausibly missing topics, retrieves candidate documents, and applies an LLM-as-judge to retain only those that introduce previously-unretrieved information. On four public benchmarks, Re:CAP recovers 9-29% of gold labels that flat BM25 top-500 cannot reach, rising to 48% on TREC-COVID. On MuSiQue Re:CAP beats flat hybrid top-500 by +12.9 pp on recall at less than half the document budget. An ensemble BM25, dense, and hybrid baseline (top-500 each) still leaves out 21.2% of gold docs on TREC-COVID that Re:CAP recovers; human annotators judge that 78.9% of those structurally distinct documents add new information to the baseline answer (Fleiss $\kappa$ = 0.79, n = 123), and 73.9% on live production traffic (n = 180). End-to-end recall is reproducible to within $\pm$1% across three independent runs, making Re:CAP a stable instrument for periodic retrieval audits.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.24122 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:48:45.029Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).