Metraipolis › archive › dc_4e8416817f7c

Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG — arXiv:2609.37469

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.37469
License (as archived)
CC0-1.0
Archived
2026-10-07T16:56:56.847Z
SHA-256 of the archived text
6070760ce4d73eea3c0cc4bf5bc18d9bf32dbd9fc0230da479a768a828053c00
Stable URL of this page
https://metraipolis.com/documento/dc_4e8416817f7c

How to cite

Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG — arXiv:2609.37469 — arXiv (en), https://arxiv.org/abs/2609.37469 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4e8416817f7c (Metraipolis archive dc_4e8416817f7c, sha256 6070760ce4d73eea3c0cc4bf5bc18d9bf32dbd9fc0230da479a768a828053c00)
BibTeX
@misc{dc4e8416817f7c,
  title = {Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG — arXiv:2609.37469},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37469},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4e8416817f7c; sha256 6070760ce4d73eea3c0cc4bf5bc18d9bf32dbd9fc0230da479a768a828053c00},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37469. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Retrieval-augmented generation (RAG) grounds large language models in external sources, but retrieved passages often name the right entities without providing the facts needed to answer. Even when instructed to abstain, 12 generators answer 40.0-99.3% of insufficient-evidence questions. Training generators to abstain ties the decision to model weights, may reward answers recalled from parametric knowledge, and still requires a full generator call. Can sufficiency be judged from the question and evidence alone, before any answer exists? We identify pitfalls in constructing insufficient-evidence tests: removing relevant evidence or pairing evidence with unrelated questions can reveal labels through lexical overlap or evidence position. We build a paired benchmark using substitution, deletion, and question-swap constructions that vary answer support while controlling selected surface features, such as word use. Sufficiency can be judged without generating an answer, but no single signal works across all datasets. We introduce RINSE (Relevance Is Not Sufficient Evidence), which combines three signals: whether every part of the question is covered, whether any passage offers an answer, and whether a small language model reading the passages together judges them sufficient. Across six datasets, RINSE ranks sufficient above insufficient evidence with a score of 0.837 (chance 0.5), exceeding the best of 10 prior methods (0.746) and a frontier model queried through an API (0.784). Its weakest dataset scores higher than any other method's weakest (0.684 vs. 0.676). RINSE runs locally before generation, taking 36.5 ms per question on a single GPU.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Information Retrieval (cs.IR); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.37469 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T16:27:03.292Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).