Metraipolis › archive › dc_c2a16fc5d6db

Where Can We Fool the Model's Mind? Response-Guided Semantic Probing and Adaptive Routing for Transferable Attacks on Black-Box MLLMs — arXiv:2610.38013

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.38013
License (as archived)
CC0-1.0
Archived
2026-11-01T02:40:16.905Z
SHA-256 of the archived text
7dfb3bc80cd0396b9f41edd055cf9e96d7394fe79c36ed334db625506d5fb22b
Stable URL of this page
https://metraipolis.com/documento/dc_c2a16fc5d6db

How to cite

Where Can We Fool the Model's Mind? Response-Guided Semantic Probing and Adaptive Routing for Transferable Attacks on Black-Box MLLMs — arXiv:2610.38013 — arXiv (en), https://arxiv.org/abs/2610.38013 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c2a16fc5d6db (Metraipolis archive dc_c2a16fc5d6db, sha256 7dfb3bc80cd0396b9f41edd055cf9e96d7394fe79c36ed334db625506d5fb22b)
BibTeX
@misc{dcc2a16fc5d6db,
  title = {Where Can We Fool the Model's Mind? Response-Guided Semantic Probing and Adaptive Routing for Transferable Attacks on Black-Box MLLMs — arXiv:2610.38013},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.38013},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c2a16fc5d6db; sha256 7dfb3bc80cd0396b9f41edd055cf9e96d7394fe79c36ed334db625506d5fb22b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.38013. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multimodal Large Language Models (MLLMs) ground language in visual evidence, yet their semantic reasoning can be exploited through transferable black-box adversarial attacks. Existing attacks typically allocate perturbations using surrogate-model saliency, which may not reflect how an unseen MLLM semantically interprets or relies on different image regions. We introduce VISTA (Vulnerability-Informed Semantic Transfer Attacks), a black-box framework that identifies semantically vulnerable visual regions from an MLLM's responses without access to gradients, logits, parameters, or internal representations. VISTA supports two attack settings: (i) VISTA-Proxy, a pure-transfer setting that probes an open-source MLLM and transfers the resulting adversarial examples to unseen commercial MLLMs without querying them; and (ii) VISTA-Victim, a query-assisted setting that probes the target MLLM to capture model-specific semantic vulnerabilities. VISTA builds an Uncertainty-Based Multimodal Behavior (UBMB) map from response drift, conflict, misleading, and semantic shifts, then uses entropy-adaptive Sinkhorn routing and a surrogate ensemble to focus perturbations on vulnerable regions. Across GPT-5, Gemini-3.5-Flash, and Claude Sonnet 4.6, VISTA achieves up to 97 % attack success rate (ASR) while maintaining a Peak Signal-to-Noise Ratio (PSNR) above 30 dB. Notably, VISTA-Proxy achieves up to 93 % ASR without querying the final victim, demonstrating that response-derived semantic vulnerability can enable effective transfer across heterogeneous MLLMs.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2610.38013 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-01T02:16:43.861Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).