Metraipolis › archive › dc_7248a446ddae

Treating the Model as an Untrusted Component: Deterministic Policy Enforcement, Human Approval and Taint Tracking Against Prompt Injection in Tool-Using Agents, with… — arXiv:2611.11571

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.11571
License (as archived)
CC0-1.0
Archived
2026-11-10T18:43:32.757Z
SHA-256 of the archived text
795abded3db324d42c4d8be798eb03992328ca87df1d82bc6a1e58ad0487be8c
Stable URL of this page
https://metraipolis.com/documento/dc_7248a446ddae

How to cite

Treating the Model as an Untrusted Component: Deterministic Policy Enforcement, Human Approval and Taint Tracking Against Prompt Injection in Tool-Using Agents, with… — arXiv:2611.11571 — arXiv (en), https://arxiv.org/abs/2611.11571 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7248a446ddae (Metraipolis archive dc_7248a446ddae, sha256 795abded3db324d42c4d8be798eb03992328ca87df1d82bc6a1e58ad0487be8c)
BibTeX
@misc{dc7248a446ddae,
  title = {Treating the Model as an Untrusted Component: Deterministic Policy Enforcement, Human Approval and Taint Tracking Against Prompt Injection in Tool-Using Agents, with… — arXiv:2611.11571},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.11571},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7248a446ddae; sha256 795abded3db324d42c4d8be798eb03992328ca87df1d82bc6a1e58ad0487be8c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.11571. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Tool-using LLM agents act on instructions embedded in the data they read, so prompt injection is a structural rather than a prompt-wording problem. We compare in-prompt defences (hardened prompts, spotlighting, LLM detectors) with deterministic enforcement outside the model (four reference monitors of increasing strength), human approval and taint tracking, judged by one effect-based oracle independent of every defence in an executable environment of 13 tasks and 12 tools, with argument-fuzzing attackers and severity-weighted harm. In an evaluation with real open-weights models (Qwen2.5-1.5B and Qwen2.5-7B; 156 attack episodes per defence and model), the models proposed the attacker's call in 64% and 84% of attack episodes behind the task-bound monitor, which admitted 1.3% and 7.7%; guardrail effects depended on the model (a prompt guardrail moved obedience from 67% to 65% and from 86% to 55%, spotlighting plus a detector to 12% and 0%) against static attacks. Simulation with an assumed model covers what no real run did: under the assumed erosion law a persistent adaptive attacker or a compromised model drives every guardrail stack to 100.0% attack success, whereas the task-bound monitor holds severe actions at 0.0%; its residual (harm 0.69/10) falls by 62% under taint-triggered human review, and a human layer without a deterministic monitor fails against approval spamming unless each episode has a deterministic approval budget. Implementation quality matters: a naive monitor admits 18 of 33 effective attack variants, a hardened one 6 and a capability-bound one 1. The real-model evidence is limited to two open-weights models and static attacks, the human-approval and taint results are simulation-only, and the larger model's detector stack contradicts the assumed erosion law. Code, data, a two-rater oracle audit and a notebook for repeating the real-model evaluation are provided.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.11571 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.650Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).