Metraipolis › archive › dc_74a244ce2495
Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment — arXiv:2609.35291
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.35291
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T22:39:00.589Z
- SHA-256 of the archived text
6ae9e461e519ebe444ce03ff4a5d029aac7c7bebf82e6b375fa2ac388403b740- Stable URL of this page
https://metraipolis.com/documento/dc_74a244ce2495
How to cite
Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment — arXiv:2609.35291 — arXiv (en), https://arxiv.org/abs/2609.35291 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_74a244ce2495 (Metraipolis archive dc_74a244ce2495, sha256 6ae9e461e519ebe444ce03ff4a5d029aac7c7bebf82e6b375fa2ac388403b740)
BibTeX
@misc{dc74a244ce2495,
title = {Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment — arXiv:2609.35291},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.35291},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_74a244ce2495; sha256 6ae9e461e519ebe444ce03ff4a5d029aac7c7bebf82e6b375fa2ac388403b740},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35291. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Modern AI models are aligned through post-training to adapt them to downstream tasks. Recent work shows that fine-tuning language models on narrow tasks can induce emergent misalignment (EM), causing broadly harmful behaviors beyond the training task. However, EM has been studied almost entirely in text-only tasks, leaving its manifestation in multimodal models unclear. In this paper, we define and analyze EM in the context of vision-language models. We first induce EM via fine-tuning on narrow multimodal tasks targeting vulnerable code, careless household-object use, and conspiratorial interpretations of ordinary scenes. Across fifteen commercial and open-source models with different scales, we find that narrow multimodal fine-tuning can induce coherent and broadly misaligned behavior that transfers to unrelated tasks, including misaligned opinions, visual factual dishonesty, unsafe image generation, vulnerability to visual jailbreaks, and risky agentic actions. We further find that multimodal EM does not depend on the apparent harmfulness of training data but is sensitive to training-evaluation modality alignment. EM can arise under both supervised fine-tuning and preference optimization and can propagate through intermediate reasoning. Finally, we explore several mitigation strategies, including prompt inoculation, benign continued training, and activation-level steering, which can partially reduce EM. Overall, our findings suggest that multimodal EM reflects a behavioral shift rather than a general loss of capability, extending beyond text to the visual modality. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG). Categorie arXiv: Machine Learning (cs.LG) ; Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.35291 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T22:14:28.405Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).