Search papers, labs, and topics across Lattice.
The paper introduces BadCLIP++, a novel framework for creating stealthy and persistent backdoor attacks in multimodal contrastive learning models, addressing limitations of existing methods related to cross-modal inconsistency and gradient dilution. BadCLIP++ enhances stealthiness by employing a semantic-fusion QR micro-trigger and target-aligned subset selection, while it improves persistence through trigger embedding stabilization (radius shrinkage and centroid alignment) and model parameter stabilization (curvature control and elastic weight consolidation). The authors provide theoretical analysis demonstrating the co-directional nature of clean fine-tuning and backdoor objectives within a trust region, and empirical results show BadCLIP++ achieves high attack success rates with minimal impact on clean accuracy, even under various defenses and in physical settings.
Even with only 0.3% data poisoning, BadCLIP++ achieves near-perfect attack success rates on multimodal models and maintains effectiveness against a wide range of defenses, highlighting a significant vulnerability.
Research on backdoor attacks against multimodal contrastive learning models faces two key challenges: stealthiness and persistence. Existing methods often fail under strong detection or continuous fine-tuning, largely due to (1) cross-modal inconsistency that exposes trigger patterns and (2) gradient dilution at low poisoning rates that accelerates backdoor forgetting. These coupled causes remain insufficiently modeled and addressed. We propose BadCLIP++, a unified framework that tackles both challenges. For stealthiness, we introduce a semantic-fusion QR micro-trigger that embeds imperceptible patterns near task-relevant regions, preserving clean-data statistics while producing compact trigger distributions. We further apply target-aligned subset selection to strengthen signals at low injection rates. For persistence, we stabilize trigger embeddings via radius shrinkage and centroid alignment, and stabilize model parameters through curvature control and elastic weight consolidation, maintaining solutions within a low-curvature wide basin resistant to fine-tuning. We also provide the first theoretical analysis showing that, within a trust region, gradients from clean fine-tuning and backdoor objectives are co-directional, yielding a non-increasing upper bound on attack success degradation. Experiments demonstrate that with only 0.3% poisoning, BadCLIP++ achieves 99.99% attack success rate (ASR) in digital settings, surpassing baselines by 11.4 points. Across nineteen defenses, ASR remains above 99.90% with less than 0.8% drop in clean accuracy. The method further attains 65.03% success in physical attacks and shows robustness against watermark removal defenses.
