Metraipolis › archive › dc_0e22af80d69e

Where Do LLMs Decide to Break the Rules? Mechanistic Localization of Prompt Injection Compliance — arXiv:2609.37737

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.37737
License (as archived)
CC0-1.0
Archived
2026-10-07T17:27:02.222Z
SHA-256 of the archived text
e2589d6db46d802cdc97fbab49d51274a1090590ca1490d467b1618b88e1a6e5
Stable URL of this page
https://metraipolis.com/documento/dc_0e22af80d69e

How to cite

Where Do LLMs Decide to Break the Rules? Mechanistic Localization of Prompt Injection Compliance — arXiv:2609.37737 — arXiv (en), https://arxiv.org/abs/2609.37737 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0e22af80d69e (Metraipolis archive dc_0e22af80d69e, sha256 e2589d6db46d802cdc97fbab49d51274a1090590ca1490d467b1618b88e1a6e5)
BibTeX
@misc{dc0e22af80d69e,
  title = {Where Do LLMs Decide to Break the Rules? Mechanistic Localization of Prompt Injection Compliance — arXiv:2609.37737},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37737},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0e22af80d69e; sha256 e2589d6db46d802cdc97fbab49d51274a1090590ca1490d467b1618b88e1a6e5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37737. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

When a prompt injection attack succeeds, a Large Language Model (LLM) abandons its assigned system role to comply with an adversarial instruction. While prior work has extensively quantified how often this occurs, we ask a more fundamental question: where inside the network does the model actually decide to break the rules? Using layer-by-layer causal activation patching across five models (4B to 32B parameters), we find a clear dissociation: attack information is linearly decodable from the first layer, yet causal leverage over the model's behavior is negligible until a late-layer bottleneck in the final third of the network. Patching this bottleneck reverses compliance in 77--92\% of cases. We show that the compliance mechanism occupies a compact linear subspace (rank-8 in 4B and 14B models, scaling to rank-64 at 32B) and is architecturally stable across varying model families. Finally, we validate our mechanistic account by showing that this causal peak layer is also the representationally optimal site for detecting attacks, outperforming early-layer classifiers that degrade under surface-level obfuscation such as leetspeak substitution. This alignment between causal leverage and detection performance provides converging evidence that the late-layer bottleneck captures decision-relevant computation rather than merely reflecting an artifact of the intervention.

Tema degli esploratori: fiducia digitale e crittografia (retro-catalogo arXiv: annunci del 2026-09-30 in cs.CR).
Categorie arXiv: Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2609.37737 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T16:58:17.366Z dalla pagina https://arxiv.org/catchup/cs.CR/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).