Metraipolis › archive › dc_64cd9c3c1306

Jev as an LLM Safety Guardrail: An Empirical Study of Input Filtering, Selective Output Review, and Over-Refusal — arXiv:2611.08845

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.08845
License (as archived)
CC0-1.0
Archived
2026-11-09T22:40:31.306Z
SHA-256 of the archived text
dbd201bc21242431f6369950f7dae5f980a8d62b46162ec852729981d59e42fe
Stable URL of this page
https://metraipolis.com/documento/dc_64cd9c3c1306

How to cite

Jev as an LLM Safety Guardrail: An Empirical Study of Input Filtering, Selective Output Review, and Over-Refusal — arXiv:2611.08845 — arXiv (en), https://arxiv.org/abs/2611.08845 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_64cd9c3c1306 (Metraipolis archive dc_64cd9c3c1306, sha256 dbd201bc21242431f6369950f7dae5f980a8d62b46162ec852729981d59e42fe)
BibTeX
@misc{dc64cd9c3c1306,
  title = {Jev as an LLM Safety Guardrail: An Empirical Study of Input Filtering, Selective Output Review, and Over-Refusal — arXiv:2611.08845},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.08845},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_64cd9c3c1306; sha256 dbd201bc21242431f6369950f7dae5f980a8d62b46162ec852729981d59e42fe},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08845. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

External safety filters can prevent unsafe requests from reaching a large language model (LLM), but may also reject legitimate queries. We present an end-to-end empirical evaluation of Jev, a probabilistic decision model, as a safety guardrail for a Qwen2.5-1.5B-based pipeline. Three configurations are compared on 970 benchmark prompts: an unfiltered LLM, Jev input filtering, and tiered Jev input filtering with selective output review. Across 720 prompts labeled harmful (520 AdvBench and 200 XSTest-unsafe), the fraction recorded as answered decreases from 527/720 (73.2%) without filtering to 17/720 (2.4%) with input filtering. On 250 XSTest-safe prompts, however, non-answer/refusal increases from 3/250 (1.2%) to 27/250 (10.8%). Tiered filtering yields 40/720 (5.6%) harmful-prompt answers and 10/250 (4.0%) benign-prompt refusals, with 79/970 (8.1%) of prompts requiring output review. Joint optimization of the tiered thresholds improves macro-F1 from 0.9351 to 0.9512, while reducing harmful-prompt answers and output-review frequency at the same benign-refusal rate. Matched-budget comparisons reveal modest additional gains over a retuned single gate on the evaluation data, although these gains are not consistently observed on a held-out split.

Tema degli esploratori: fiducia digitale e crittografia (elenco arXiv cs.CR, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2611.08845 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-09T22:33:17.028Z dalla pagina https://arxiv.org/list/cs.CR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).