Search papers, labs, and topics across Lattice.
This paper introduces SafeCA, a novel defense mechanism designed to protect text-to-video generative models from jailbreak attacks by analyzing cross-attention feature spaces. By identifying key defensive regions and employing attention masking alongside a semantic-space adapter, SafeCA effectively mitigates harmful activations while maintaining semantic integrity. Experimental results demonstrate that SafeCA reduces jailbreak success rates by approximately 20% with minimal inference overhead, marking a significant advancement in the safety of T2V models.
SafeCA slashes jailbreak success rates by 20% while adding virtually no latency, revolutionizing defenses for text-to-video models.
Text-to-Video (T2V) generative models are vulnerable to jailbreak attacks in real-world deployment, leading them to produce harmful or inappropriate content. Existing defense approaches mainly rely on input filtering or reconstruction, which not only incur high computational latency but also tend to distort semantics. To address these issues, we experimentally and systematically analyze the differences between clean and jailbreak samples in the cross-attention feature space, revealing for the first time a cumulative separation effect and a progressively increasing trend of linear separability between the two during the diffusion process. Based on this insight, we propose SafeCA, a feature-level defense mechanism for safe cross-attention localization and regularization. Firstly, we identify key defensive regions and values through attention stability analysis using cross-attention features collected from clean prompts within a single inference. Secondly, SafeCA mitigates anomalous activations via attention masking with energy normalization and introduces a lightweight semantic-space adapter to redirect abnormal semantic flows. Furthermore, we detect and suppress potentially malicious tokens by back-propagating feature anomaly signals to the input cue words, thereby enhancing the deployability of the defense in commercial models. Experimental results show that SafeCA reduces the jailbreak success rate by about 20% on mainstream T2V models, adds almost no inference overhead (+0.1s), and maintains good text-video semantic consistency. Overall, SafeCA provides an architecture-level, deployable protection paradigm for T2V generation models.
NUS