Metraipolis › archive › dc_c2b265d4ed07
The Geometry of Harmfulness in Multi-Turn Attacks — arXiv:2609.38389
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.38389
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T13:14:16.881Z
- SHA-256 of the archived text
eb094efd99de2a5c80ef5824439fa6eea4e77011452b7e1e9e6b23c005ada682- Stable URL of this page
https://metraipolis.com/documento/dc_c2b265d4ed07
How to cite
The Geometry of Harmfulness in Multi-Turn Attacks — arXiv:2609.38389 — arXiv (en), https://arxiv.org/abs/2609.38389 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c2b265d4ed07 (Metraipolis archive dc_c2b265d4ed07, sha256 eb094efd99de2a5c80ef5824439fa6eea4e77011452b7e1e9e6b23c005ada682)
BibTeX
@misc{dcc2b265d4ed07,
title = {The Geometry of Harmfulness in Multi-Turn Attacks — arXiv:2609.38389},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.38389},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c2b265d4ed07; sha256 eb094efd99de2a5c80ef5824439fa6eea4e77011452b7e1e9e6b23c005ada682},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38389. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) remain vulnerable to adversarial attacks that circumvent safety alignment to elicit harmful outputs. It remains unclear how harmfulness and refusal representations evolve over the course of multi-turn attacks, and why single-turn defenses are less effective in multi-turn settings. This work investigates how the geometry and temporal dynamics of harmfulness and refusal representations evolve across multi-turn attacks. We analyzed hidden-state representations from three instruction-tuned LLMs (Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-it) using three multi-turn attack frameworks (Crescendo, ActorAttack, and X-Teaming), and examined representation behavior across conversation turns, model layers, and token positions under various context configurations. Across models and frameworks, we found that (1) each attack framework traverses different geometric directions, yet each achieves comparable success in eliciting harmful outputs; (2) multi-turn harmfulness directions became increasingly linearly separable at the end-of-turn token position across turns in middle to late model layers; and (3) harmfulness representations are weakly aligned with refusal-related representations. The results indicate that multi-turn attacks do not succeed by suppressing the model's internal representation of harmfulness. Instead, harmfulness representations become increasingly separable across conversation turns, while remaining only weakly aligned with refusal-related representations. The findings are one possible explanation for why static single-turn safety probes may degrade in multi-turn settings, and suggest that robust defenses must consider temporal representation dynamics rather than identifying harmfulness with isolated or single-turn prompts. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.CL). Categorie arXiv: Cryptography and Security (cs.CR) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.38389 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T12:44:51.927Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).