Metraipolis › archive › dc_b3ec416a1c19

PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift — arXiv:2609.15017

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.15017
License (as archived): CC0-1.0
Archived: 2026-10-15T22:53:30.950Z
SHA-256 of the archived text: 609cac9d32a21c7398310e9feaf6d30a43b0cfeafebe63f36fe0524bfe3ab31d
Stable URL of this page: https://metraipolis.com/documento/dc_b3ec416a1c19

How to cite

PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift — arXiv:2609.15017 — arXiv (en), https://arxiv.org/abs/2609.15017 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b3ec416a1c19 (Metraipolis archive dc_b3ec416a1c19, sha256 609cac9d32a21c7398310e9feaf6d30a43b0cfeafebe63f36fe0524bfe3ab31d)

BibTeX

@misc{dcb3ec416a1c19,
  title = {PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift — arXiv:2609.15017},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.15017},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b3ec416a1c19; sha256 609cac9d32a21c7398310e9feaf6d30a43b0cfeafebe63f36fe0524bfe3ab31d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.15017. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Prompt-injection detectors are typically evaluated using aggregate F1 on in-distribution test data, which offers limited insight into behavior under distribution shift, particularly on the benign side of the decision boundary, where false positives impose direct operational cost yet are seldom measured. We present PIDS-Bench, a frozen multi-axis benchmark that jointly evaluates attack detection and benign false-positive behavior at fixed thresholds, spanning in-distribution inputs, hard-benign prompts that mimic injection structure without malicious intent, obfuscated attacks, and domain and structural distribution shifts. We evaluate seven detectors (learned baselines, external prompt-injection classifiers, and broad-safety comparators) alongside a rule-based lower-bound reference. Multi-axis evaluation exposes a failure mode that aggregate F1 conceals. A detector exceeding F1 = 0.98 on the held-out split still misclassifies roughly one-third of an externally-sourced benign subset drawn from public corpora and restricted to security-adjacent content. Across a full threshold sweep and five training seeds, no internal detector reaches an operating point satisfying F1 >= 0.95 and hard-benign FPR <= 0.10 together on this stress distribution. Decomposing by provenance, we find that hard-negative augmentation nearly eliminates over-defense on curated stress inputs but leaves it substantially intact on externally-sourced prompts, a pattern we term provenance-sensitive over-defense. The asymmetry holds across both fine-tuned architectures and does not diminish as the augmentation pool grows, with the externally-sourced FPR remaining far above the 0.10 target. Whether augmentation matched to the externally-sourced distribution would close this gap is untested; threshold calibration and curated-style augmentation alone do not.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.AI).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.15017 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T22:29:03.774Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).