Metraipolis › archive › dc_34fd3beb584e
Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue — arXiv:2609.36849
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36849
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:26:26.655Z
- SHA-256 of the archived text
b013d2f265518267f9d362815787f2e91d5e31a2bd0a29e3555491d95f751dd2- Stable URL of this page
https://metraipolis.com/documento/dc_34fd3beb584e
How to cite
Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue — arXiv:2609.36849 — arXiv (en), https://arxiv.org/abs/2609.36849 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_34fd3beb584e (Metraipolis archive dc_34fd3beb584e, sha256 b013d2f265518267f9d362815787f2e91d5e31a2bd0a29e3555491d95f751dd2)
BibTeX
@misc{dc34fd3beb584e,
title = {Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue — arXiv:2609.36849},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36849},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_34fd3beb584e; sha256 b013d2f265518267f9d362815787f2e91d5e31a2bd0a29e3555491d95f751dd2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36849. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Safety-aligned language models are commonly deployed as multi-turn assistants, which lets adversaries spread unsafe intent across several user turns instead of a single prompt. Gradient-based jailbreak detectors such as GradSafe were developed for single prompts: they score an input by the alignment between its induced gradient and a fixed unsafe reference direction, and their effectiveness in multi-turn dialogue remains unclear. We conduct a controlled evaluation of gradient-based jailbreak detection in multi-turn settings. We extend GradSafe with a Context Window Scanner that applies the detector to fixed-size windows of user turns and uses the maximum window score as the conversation-level score. We evaluate different window sizes, attack families, benign conversation distributions, and target models. The results differ sharply between synthetic and realistic benign settings. Against synthetic benign conversations, the detector achieves an ROC-AUC of 0.98 on human-authored multi-turn jailbreaks. On WildChat benign conversations, ROC-AUC drops to 0.76, and a threshold calibrated on synthetic data flags more than 90% of benign conversations as unsafe. Under realistic benign distributions, single-turn windows give the highest separability, whereas longer windows and accumulated contexts reduce performance. The detector is also sensitive to the attack-generation method and target model: successful Crescendo attacks receive scores comparable to or lower than benign conversations, and Qwen2.5-7B-Instruct yields near-random separability with a different optimal window size. These findings show that gradient-based signals can support multi-turn jailbreak detection, but reliable deployment requires calibration on realistic benign conversations, short-window scoring, length-aware thresholds, and evaluation across attack types and model architectures. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI). Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.36849 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T15:54:53.897Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).