Metraipolis › archive › dc_336a9de06154

Ranked but Not Stopped: Detecting Prompt-Injection Hijacks from Typed Decision Models' Output Probabilities — arXiv:2611.02522

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02522
License (as archived)
CC0-1.0
Archived
2026-11-03T18:23:41.509Z
SHA-256 of the archived text
d4d4105e7e54b34dfc1987d7701cfba7c867e16b647303694d31640002fdd284
Stable URL of this page
https://metraipolis.com/documento/dc_336a9de06154

How to cite

Ranked but Not Stopped: Detecting Prompt-Injection Hijacks from Typed Decision Models' Output Probabilities — arXiv:2611.02522 — arXiv (en), https://arxiv.org/abs/2611.02522 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_336a9de06154 (Metraipolis archive dc_336a9de06154, sha256 d4d4105e7e54b34dfc1987d7701cfba7c867e16b647303694d31640002fdd284)
BibTeX
@misc{dc336a9de06154,
  title = {Ranked but Not Stopped: Detecting Prompt-Injection Hijacks from Typed Decision Models' Output Probabilities — arXiv:2611.02522},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02522},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_336a9de06154; sha256 d4d4105e7e54b34dfc1987d7701cfba7c867e16b647303694d31640002fdd284},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02522. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Programs that use a language model as a classifier act on its most probable label. Text injected into the input can move that label to one the attacker chooses, a hijack. A monitor that does not know the attacker's label can read the model's own confidence and raise an alarm when the top probability is low. We test this monitor on AG News topic classification with five systems, three of them served through a provider's API. Pooled over five families of injected text, low confidence ranks hijacked answers above clean ones on four of the five systems (AUROC 0.86-0.90). Ranking them does not stop many of them. A threshold set so that at most 5% of clean answers to separate calibration articles alarm falls among the least confident clean answers, and every more confident hijack passes. In our main gate run on the three API-served systems, the monitor misses 51-70% of the hijacks caused by a plain injected note (per-system point estimates). When the note adds one handwritten sentence that calls the target certain and asks for full probability, the monitor misses 99-100% of the hijacks, although low confidence still ranks them at AUROC 0.70-0.79. Confidence monitors should be tested at their operating alarm rate, against attackers who claim certainty.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2611.02522 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:10:07.970Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).