Metraipolis › archive › dc_fcda55daae47

How Fragile Is Safety Alignment at Frontier Scale? A Single-Direction Attack on a 320B MoE — arXiv:2609.09793

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.09793
License (as archived): CC0-1.0
Archived: 2026-10-16T07:26:23.411Z
SHA-256 of the archived text: 21ef5ed1c7e754af0466e829dfc84fc85c8bb6e517634f7516d95c71baecc2d8
Stable URL of this page: https://metraipolis.com/documento/dc_fcda55daae47

How to cite

How Fragile Is Safety Alignment at Frontier Scale? A Single-Direction Attack on a 320B MoE — arXiv:2609.09793 — arXiv (en), https://arxiv.org/abs/2609.09793 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fcda55daae47 (Metraipolis archive dc_fcda55daae47, sha256 21ef5ed1c7e754af0466e829dfc84fc85c8bb6e517634f7516d95c71baecc2d8)

BibTeX

@misc{dcfcda55daae47,
  title = {How Fragile Is Safety Alignment at Frontier Scale? A Single-Direction Attack on a 320B MoE — arXiv:2609.09793},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.09793},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fcda55daae47; sha256 21ef5ed1c7e754af0466e829dfc84fc85c8bb6e517634f7516d95c71baecc2d8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09793. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Directional ablation removes an aligned language model's ability to refuse by projecting a single "refusal direction" out of the weights that write the residual stream. It needs no gradient-based training and no optimization, only a few hundred contrastive prompts, which makes it the canonical white-box attack on open-weight alignment. However, it has been established only on dense models up to roughly 70B parameters. We study whether it survives the shift to frontier mixture-of-experts (MoE) models whose residual streams are no longer a single tensor and whose weights ship quantized. We apply it to GLM-5.3-Flash (320B parameters, 288 routed experts, a four-wide hyper-connection residual, block-FP8). The attack survives the architecture, but what it reaches is no longer where a reader of the original recipe would look for it. Editing the attention, dense and routed-expert writers on their own removes 0.039, 0.016 and 0.148 of refusal respectively; editing all three together removes 0.776. As a result, 74% of the effect exists only under the joint intervention. The part the conventional recipe reaches by module-name matching accounts for 0.066 of that 0.776, which is why it fails silently on an MoE. The effect does not follow from removing just any direction: ablating a random direction orthogonal to it leaves refusal unchanged. A category-concentrated residue survives every edit we tried: subspaces fitted on violence, sexual content and hate leave measurable refusal at every rank from 1 to 12. We report the method, the 41-89 percentage-point reductions it achieves across seven harmful benchmarks with no detected change in capability, and the boundary where it stops.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-10 in cs.AI).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.09793 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T07:00:16.350Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).