Metraipolis › archive › dc_a616111bbee7

Breaking the weakest link to evade vision language models — arXiv:2608.18938

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.18938
License (as archived)
CC0-1.0
Archived
2026-10-18T13:44:40.297Z
SHA-256 of the archived text
cb6db8aa86560343d14e0f7b27a602f0839e0d1f68b271ec653beb9b78f5a08d
Stable URL of this page
https://metraipolis.com/documento/dc_a616111bbee7

How to cite

Breaking the weakest link to evade vision language models — arXiv:2608.18938 — arXiv (en), https://arxiv.org/abs/2608.18938 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a616111bbee7 (Metraipolis archive dc_a616111bbee7, sha256 cb6db8aa86560343d14e0f7b27a602f0839e0d1f68b271ec653beb9b78f5a08d)
BibTeX
@misc{dca616111bbee7,
  title = {Breaking the weakest link to evade vision language models — arXiv:2608.18938},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.18938},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a616111bbee7; sha256 cb6db8aa86560343d14e0f7b27a602f0839e0d1f68b271ec653beb9b78f5a08d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.18938. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Vision Language Models (VLMs) have recently emerged as a critical component of multimodal AI systems, enabling joint reasoning over visual and textual inputs in real-world and safety-critical applications. Despite their growing deployment, the robustness of VLMs against adversarial threats remains insufficiently explored, particularly in the context of evasion attacks targeting multimodal alignment. In this work, we investigate the vulnerability of VLMs to adversarial perturbations applied to visual inputs and study two attack settings: untargeted attacks, where the goal is to disrupt the model's interpretation of the original image, and targeted attacks, where the adversary aims to force the model to generate a specific semantic description unrelated to the original image. To efficiently generate adversarial examples, we propose a gradient-based attack method that performs optimization exclusively on the vision encoder of the VLM rather than on the entire multimodal architecture. This design significantly reduces the computational cost and resource requirements of the attack while maintaining strong effectiveness. We evaluate our approach on several open-source VLMs, including Qwen2.5-VL, Granite-Vision, FastVLM, and Phi-3.5-Vision, and show that small, human-imperceptible perturbations can substantially alter the textual interpretation produced by the models. Our findings highlight the vulnerability of modern VLMs to adversarial manipulation and emphasize the need for improved robustness and security mechanisms in multimodal AI systems.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.18938 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T13:41:17.887Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).