Search papers, labs, and topics across Lattice.
This paper introduces RACER, a model-level framework designed to repair backdoors in multimodal large language models (MLLMs) by addressing the layer-wise inconsistency anomalies induced by backdoor triggers. By decomposing representations into visual and textual token regions and applying modality-aware normalization, RACER effectively suppresses backdoor behaviors while maintaining model performance. Evaluations demonstrate that RACER significantly reduces the average attack success rate (ASR) to 1.1%, achieving 0% ASR in 32 out of 36 tested scenarios, all while preserving utility on clean tasks.
Backdoor attacks in MLLMs can be effectively neutralized without prior knowledge of the trigger, achieving near-perfect defense in most scenarios.
MLLMs are increasingly deployed in user-facing applications, yet they inherit backdoor risks from the pipelines used to construct them: triggers may reside in images, texts, or both. Existing model-level backdoor removal methods, largely designed for conventional classifiers, show limited effectiveness on MLLMs, while MLLM-specific defenses mainly operate at inference time, filtering suspicious inputs without removing the backdoor embedded in the model. To address this gap and eliminate latent backdoors from MLLMs at their source, we present RACER, a model-level repair framework motivated by a key observation: backdoors induce abnormal layer-to-layer evolution in internal representations, which we term the layer-wise inconsistency anomaly. Importantly, this anomaly is modality-dependent, concentrating primarily in the token region encoding the trigger features that the backdoor model actually relies on. RACER therefore decomposes the fused representation into visual and textual token regions, normalizes their layer-wise inconsistency separately, and recomposes them using modality-aware weights over a deep-layer window, yielding a region-aware inconsistency objective that better captures localized backdoor-induced anomalies. Through a min-max optimization, this objective drives worst-case perturbation synthesis and adversarial fine-tuning against the resulting perturbation to repair the model, suppressing the deep representational directional shifts on which backdoor behaviors rely. RACER requires only 100 clean samples and no knowledge of the trigger, attack objective, or even whether the input model contains a backdoor. Evaluations on three open-source MLLMs across 36 backdoor settings spanning image, text, and multimodal triggers show that RACER reduces the average ASR to 1.1%, reaching 0% in 32 settings, while preserving clean-task utility on both backdoor and clean models.
