Metraipolis › archive › dc_369d18e954ba

Mitigating Reasoning-Induced Misalignment via Safety-Direction Penalty — arXiv:2608.23497

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.23497
License (as archived)
CC0-1.0
Archived
2026-10-18T07:41:59.530Z
SHA-256 of the archived text
96a8b6237de51a27ef32db75efa84b8ffb1f6086044546eb3d616fcf6b67f79d
Stable URL of this page
https://metraipolis.com/documento/dc_369d18e954ba

How to cite

Mitigating Reasoning-Induced Misalignment via Safety-Direction Penalty — arXiv:2608.23497 — arXiv (en), https://arxiv.org/abs/2608.23497 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_369d18e954ba (Metraipolis archive dc_369d18e954ba, sha256 96a8b6237de51a27ef32db75efa84b8ffb1f6086044546eb3d616fcf6b67f79d)
BibTeX
@misc{dc369d18e954ba,
  title = {Mitigating Reasoning-Induced Misalignment via Safety-Direction Penalty — arXiv:2608.23497},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.23497},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_369d18e954ba; sha256 96a8b6237de51a27ef32db75efa84b8ffb1f6086044546eb3d616fcf6b67f79d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.23497. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reasoning-Induced Misalignment, where fine-tuning on reasoning data containing no harmful content, including mathematics, code, and problem-solving with chain-of-thought traces can induce harmful behaviors of LLM, posing a serious challenge to the safety of LLM reasoning. Cross-architecture, cross-scale, and cross-dataset checks show that RIM does not always emerge. Previous work attributed RIM to neuron-level entanglement, but did not identify the geometry of the representation space underlying this entanglement or propose a training-time fix. We provide both: a representation-space analysis of RIM and the Safety-Direction Penalty (SDP), which penalizes movement along a learned safety direction during reasoning fine-tuning. The analysis extracts two activation-space directions, one encoding reasoning ability and the other safety behavior. These directions are coupled: fine-tuning that improves reasoning shifts safety representations, and prompts with larger shifts show larger safety degradation. CKA distance ratios and probes locate the safety-decision layers where this shift is most relevant. These findings guide the design of SDP: the coupling motivates penalizing displacement along the safety direction, and the layer localization sets the initial scope. When the initial scope leaves compensatory shifts beyond the penalized layers, the same diagnostics guide iterative expansion. On Qwen2.5-3B and 7B, SDP restores safety while preserving benchmark reasoning performance.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-25 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.23497 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T07:37:52.782Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).