Metraipolis › archive › dc_39f7fe75013f

Ranking Is Not Deciding: Benchmark Shortcuts and Operating-Point Collapse in Query-Time RAG Poisoning Defenses — arXiv:2610.33849

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.33849
License (as archived)
CC0-1.0
Archived
2026-10-27T08:12:38.476Z
SHA-256 of the archived text
e3097814e9378f7799e2d56561d210e78fc19665e969ad11162badaf1a8a722c
Stable URL of this page
https://metraipolis.com/documento/dc_39f7fe75013f

How to cite

Ranking Is Not Deciding: Benchmark Shortcuts and Operating-Point Collapse in Query-Time RAG Poisoning Defenses — arXiv:2610.33849 — arXiv (en), https://arxiv.org/abs/2610.33849 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_39f7fe75013f (Metraipolis archive dc_39f7fe75013f, sha256 e3097814e9378f7799e2d56561d210e78fc19665e969ad11162badaf1a8a722c)
BibTeX
@misc{dc39f7fe75013f,
  title = {Ranking Is Not Deciding: Benchmark Shortcuts and Operating-Point Collapse in Query-Time RAG Poisoning Defenses — arXiv:2610.33849},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.33849},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_39f7fe75013f; sha256 e3097814e9378f7799e2d56561d210e78fc19665e969ad11162badaf1a8a722c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.33849. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Query-time filters that screen each retrieved document before it reaches the generator are the dominant published defense against retrieval-augmented generation poisoning, and they report near-perfect detection on domain benchmarks. We show that this performance is predicted by how the poison set was built rather than by what the filter reads: removing a single construction cue destroys the deployed decision before it destroys the ranking. Two label-free statistics of a poison set and its clean corpus profile the cue: containment, how often the target question appears verbatim in the poison, and length overlap. PoisonedRAG's released code prepends the target question to every inserted passage. Of the 21 published defense evaluations we survey, 8 verifiably run poison built this way and 5 substitute a length cue; separately, 12 report no false-positive rate at the operating point they deploy. Deleting only that prefix from the attack authors' own passages (0.93 still retrieved for their target) takes RAGuard, a published content filter we re-implement, from AUROC 0.953 to 0.758 and its miss rate from 0.191 to 0.763. Across four corpora, four attack families and ten per-document defenses on one shared candidate set, the profile predicts where detection saturates. Padding medical poison to clean length with word-shuffled filler that carries no content takes TrustRAG's set-level stage from a miss rate of 0.000 to 0.827, and one learned scorer of ours from 0.000 to 0.986 (0.321 at a matched 5% false-positive rate) while its AUROC holds at 0.953. The ranking survives; the decision does not. A defense-aware attacker defeats our own scorer, so we claim no defense. A detection result is evidence only when it is reported with its poison set's construction profile and the false-positive rate at the threshold actually deployed; we release the script that computes the profile.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.33849 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:56:00.297Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).