Metraipolis › archive › dc_98e64426ce79

From Detection to Decision: Calibration and Deferral for Prompt-Injection Detection — arXiv:2610.20907

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.20907
License (as archived)
CC0-1.0
Archived
2026-10-16T10:19:14.540Z
SHA-256 of the archived text
46ed1b5d523df49b0ae06f29cd3902c906475047824a370216d001fb5b9101a7
Stable URL of this page
https://metraipolis.com/documento/dc_98e64426ce79

How to cite

From Detection to Decision: Calibration and Deferral for Prompt-Injection Detection — arXiv:2610.20907 — arXiv (en), https://arxiv.org/abs/2610.20907 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_98e64426ce79 (Metraipolis archive dc_98e64426ce79, sha256 46ed1b5d523df49b0ae06f29cd3902c906475047824a370216d001fb5b9101a7)
BibTeX
@misc{dc98e64426ce79,
  title = {From Detection to Decision: Calibration and Deferral for Prompt-Injection Detection — arXiv:2610.20907},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.20907},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_98e64426ce79; sha256 46ed1b5d523df49b0ae06f29cd3902c906475047824a370216d001fb5b9101a7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.20907. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Prompt-injection defenses are commonly evaluated as binary detectors, although deployed LLM applications must make operational decisions about whether to allow, block, or escalate potentially malicious inputs. Recent probabilistic decision models aim to support such decisions through calibrated confidence estimates, but their reliability under distribution shift remains uncertain. We evaluate a native probabilistic decision model (Jev), two Llama Prompt Guard 2 classifiers (22M and 86M), and an LLM-based judge on 47,611 held-out examples spanning direct and indirect prompt injections, hard-benign inputs, obfuscation, and distribution shift. Using separate development and calibration splits, we assess detection performance, probability calibration, and allow-block-defer policies under a frozen evaluation protocol. The LLM judge achieves the highest overall discrimination (AUROC 0.897), followed by Jev (0.871), Prompt Guard 2 86M (0.757), and Prompt Guard 2 22M (0.714). However, Jev's native probabilities do not consistently outperform post-hoc calibrated classifier scores, and temperature scaling improves calibration without improving discrimination. At an empirical false-negative-rate target of 1%, all evaluated gate-reviewer systems require more than 91.8% escalation and incur substantial false-positive rates. These findings demonstrate that probabilistic outputs alone do not establish reliable security decisions under distribution shift. Effective LLM security systems must jointly evaluate detection, calibration, reviewer behavior, and operational decision policies.

Tema degli esploratori: fiducia digitale e crittografia (elenco arXiv cs.CR, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2610.20907 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:47:32.584Z dalla pagina https://arxiv.org/list/cs.CR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).