Metraipolis › archive › dc_d97384b2000f

SecPO: Principled Adversarial Training for Prompt Injection Security — arXiv:2610.41998

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41998
License (as archived): CC0-1.0
Archived: 2026-11-03T07:13:28.603Z
SHA-256 of the archived text: 52fb4b8723e5aaa09798e4e3a9d1d8d1fd7e75d937191883c0b84b926ec7b27f
Stable URL of this page: https://metraipolis.com/documento/dc_d97384b2000f

How to cite

SecPO: Principled Adversarial Training for Prompt Injection Security — arXiv:2610.41998 — arXiv (en), https://arxiv.org/abs/2610.41998 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d97384b2000f (Metraipolis archive dc_d97384b2000f, sha256 52fb4b8723e5aaa09798e4e3a9d1d8d1fd7e75d937191883c0b84b926ec7b27f)

BibTeX

@misc{dcd97384b2000f,
  title = {SecPO: Principled Adversarial Training for Prompt Injection Security — arXiv:2610.41998},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41998},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d97384b2000f; sha256 52fb4b8723e5aaa09798e4e3a9d1d8d1fd7e75d937191883c0b84b926ec7b27f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41998. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recognized as a top threat to AI agents, prompt injection allows instructions from untrusted data (email, documents, websites) to manipulate the agent's operation. The existing best-defended LLM, Meta-SecAlign, is trained with the DPO loss but still suffers from attack success rates (ASRs) approaching 100% under adaptive prompt injections. One reason is that DPO anchors the model around the undefended reference model, which is insecure. We show an idealized reference model---an injection-invariant, utility-preserving oracle---can nevertheless be approximated, as its output distribution in training is interestingly accessible. This new observation motivates a theoretically principled SecPO loss. Another Meta-SecAlign shortcoming is that it trains on weak simulated attacks. We implement an efficient adversarial training (AT) pipeline that synthesizes strong injected inputs (best@240), and train the model on them. Our AT reduces ASRs by 1.5-14.5$\times$ across three SoTA adaptive attacks, compared with Meta-SecAlign; the SecPO loss reduces them by a further 2.4-4.6$\times$ compared with using DPO loss. In agentic workflows, our SecPO + AT Qwen3.6-27B is much more secure than Meta-SecAlign: 8.8% vs. 78.8% AgentDojo ASR@800 under the AttackerMovesSecond attack. Moreover, our defended model shows no noticeable reduction in benign utility in nine benchmarks covering agentic tool calling, instruction following, science, and math. We release our $\href{//github.com/Sizhe-Chen/SecPO}{\text{code}}$, $\href{//huggingface.co/datasets/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO-trainset}{\text{training data}}$ (with optimized injections from AT), and $\href{//huggingface.co/Sizhe-Chen/Qwen3.6-27B-Instruct-SecPO}{\text{model}}$.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.41998 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T07:09:31.887Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).