Metraipolis › archive › dc_dd460a794602
Not All Tokens Are Equal: Region-Aware Consistency Repair of Backdoors in MLLMs — arXiv:2608.24354
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.24354
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T06:20:55.833Z
- SHA-256 of the archived text
6c1f0626c0d94c274daf5bc10b9a0b91c43f2dd738a234c97ef28c88d5edb29a- Stable URL of this page
https://metraipolis.com/documento/dc_dd460a794602
How to cite
Not All Tokens Are Equal: Region-Aware Consistency Repair of Backdoors in MLLMs — arXiv:2608.24354 — arXiv (en), https://arxiv.org/abs/2608.24354 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dd460a794602 (Metraipolis archive dc_dd460a794602, sha256 6c1f0626c0d94c274daf5bc10b9a0b91c43f2dd738a234c97ef28c88d5edb29a)
BibTeX
@misc{dcdd460a794602,
title = {Not All Tokens Are Equal: Region-Aware Consistency Repair of Backdoors in MLLMs — arXiv:2608.24354},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.24354},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dd460a794602; sha256 6c1f0626c0d94c274daf5bc10b9a0b91c43f2dd738a234c97ef28c88d5edb29a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.24354. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
MLLMs are increasingly deployed in user-facing applications, yet they inherit backdoor risks from the pipelines used to construct them: triggers may reside in images, texts, or both. Existing model-level backdoor removal methods, largely designed for conventional classifiers, show limited effectiveness on MLLMs, while MLLM-specific defenses mainly operate at inference time, filtering suspicious inputs without removing the backdoor embedded in the model. To address this gap and eliminate latent backdoors from MLLMs at their source, we present RACER, a model-level repair framework motivated by a key observation: backdoors induce abnormal layer-to-layer evolution in internal representations, which we term the layer-wise inconsistency anomaly. Importantly, this anomaly is modality-dependent, concentrating primarily in the token region encoding the trigger features that the backdoor model actually relies on. RACER therefore decomposes the fused representation into visual and textual token regions, normalizes their layer-wise inconsistency separately, and recomposes them using modality-aware weights over a deep-layer window, yielding a region-aware inconsistency objective that better captures localized backdoor-induced anomalies. Through a min-max optimization, this objective drives worst-case perturbation synthesis and adversarial fine-tuning against the resulting perturbation to repair the model, suppressing the deep representational directional shifts on which backdoor behaviors rely. RACER requires only 100 clean samples and no knowledge of the trigger, attack objective, or even whether the input model contains a backdoor. Evaluations on three open-source MLLMs across 36 backdoor settings spanning image, text, and multimodal triggers show that RACER reduces the average ASR to 1.1%, reaching 0% in 32 settings, while preserving clean-task utility on both backdoor and clean models. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-26 in cs.CL). Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.24354 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T06:17:29.411Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-26?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).