Metraipolis › archive › dc_4a4e56953c36

Once Poisoned, Arbitrarily Controlled: A Programmable Backdoor in VLMs — arXiv:2608.10959

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.10959
License (as archived)
CC0-1.0
Archived
2026-10-19T06:18:31.308Z
SHA-256 of the archived text
043990bccf788e45f7f3ee7b72306ec3d3c23b8309d3d3ae9caab58eda7799cc
Stable URL of this page
https://metraipolis.com/documento/dc_4a4e56953c36

How to cite

Once Poisoned, Arbitrarily Controlled: A Programmable Backdoor in VLMs — arXiv:2608.10959 — arXiv (en), https://arxiv.org/abs/2608.10959 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4a4e56953c36 (Metraipolis archive dc_4a4e56953c36, sha256 043990bccf788e45f7f3ee7b72306ec3d3c23b8309d3d3ae9caab58eda7799cc)
BibTeX
@misc{dc4a4e56953c36,
  title = {Once Poisoned, Arbitrarily Controlled: A Programmable Backdoor in VLMs — arXiv:2608.10959},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.10959},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4a4e56953c36; sha256 043990bccf788e45f7f3ee7b72306ec3d3c23b8309d3d3ae9caab58eda7799cc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.10959. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Existing vision-language model (VLM) backdoors are usually treated as static vulnerabilities: one-to-one and N-to-N attacks bind one or more triggers to a finite set of targets before victim training. This assumption substantially underestimates the threat. We show that a single poisoning phase can implant a programmable backdoor into a VLM, allowing an attacker to choose previously unseen target-caption semantics at inference time and synthesize corresponding stealthy triggers on demand. Unlike fixed-mapping attacks, the proposed any-to-any caption-control paradigm decouples post-training target selection from poisoning, enabling dynamic control of target captions without retraining the VLM. Our method has two components. First, a heuristic poisoning strategy exposes the model to diverse trigger-caption pairs, encouraging it to learn a general trigger-as-instruction rule rather than memorize a specific backdoor pattern. Second, a feature-space trigger steganography method maps any attacker-specified target caption to a stealthy visual trigger, implemented as either a norm-controlled perturbation or a non-semantic patch. Once inserted into arbitrary images, these triggers cause the poisoned VLM to generate outputs semantically aligned with the chosen target caption, even when the target was unseen during poisoning. Extensive experiments show that our attack achieves high any-to-any caption-control success rates, preserves clean model utility, and remains effective under several classical backdoor defenses.

Tema degli esploratori: fiducia digitale e crittografia (retro-catalogo arXiv: annunci del 2026-08-12 in cs.CR).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2608.10959 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T06:11:08.396Z dalla pagina https://arxiv.org/catchup/cs.CR/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).