Metraipolis › archive › dc_69727b9a97bc

RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety — arXiv:2609.11758

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.11758
License (as archived)
CC0-1.0
Archived
2026-10-16T05:44:11.897Z
SHA-256 of the archived text
3282d01175fbb2b20a2954b170af13d925b8e8f2b03e97fb262e69d8cab0f0b7
Stable URL of this page
https://metraipolis.com/documento/dc_69727b9a97bc

How to cite

RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety — arXiv:2609.11758 — arXiv (en), https://arxiv.org/abs/2609.11758 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_69727b9a97bc (Metraipolis archive dc_69727b9a97bc, sha256 3282d01175fbb2b20a2954b170af13d925b8e8f2b03e97fb262e69d8cab0f0b7)
BibTeX
@misc{dc69727b9a97bc,
  title = {RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety — arXiv:2609.11758},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.11758},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_69727b9a97bc; sha256 3282d01175fbb2b20a2954b170af13d925b8e8f2b03e97fb262e69d8cab0f0b7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11758. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Allowing large language models (LLMs) to retrieve information from a set of trusted documents can increase reliability and reduce hallucination. However, recent work has demonstrated that retrieval-augmented generation (RAG) can have unintended side effects on the overall safety of the generated responses, when prompted for harmful or dangerous content. A clearer understanding of the mechanisms leading to this result is needed, as increasing numbers of end users turn to RAG to incorporate corporate documents and knowledge bases into LLM-based systems. We introduce RAG-Safety-Bench, a benchmark to measure the safety impact of RAG on LLM models. By removing the confounding effect of retriever quality, and cleanly separating the problem into four conditions -- non-RAG, RAG with an oracle document containing the answer to the harmful request, RAG with documents related to the harmful request but without the specific answer, and RAG with random, safe documents -- the benchmark isolates the impacts of different factors in the observed safety degradation. We report results across five open-source LLMs, showing an inverse relationship between benign and unsafe capability, strong evidence that baseline safety guardrails do not lead to downstream safety guarantees in the RAG case, and model-specific support for previous findings that even benign documents can lead to unsafe generation in retrieval-enabled systems.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-11 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2609.11758 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T05:23:57.020Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).