Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors introduce DefTEval to systematically evaluate whether eleven input-level pixel defenses can mitigate five observation-level adversarial attack types across five VideoLLMs. The study reveals that input-level defenses fail catastrophically—yielding harmful content detection rates near zero even when perturbations exist across every sampled frame—because downstream token compression purges localized features and cross-modal attention systematically down-weights visual signals. These results indicate that defense transferability is governed primarily by VideoLLM architecture rather than pixel defense design, exposing critical blind spots in multimodal temporal reasoning.
Standard visual safety defenses yield near-zero detection against observation-level video jailbreaks, undone not by sparse frame sampling but by the model's own token compression and modality fusion actively discarding surviving safety signals.
Video Large Language Models (VideoLLMs) are increasingly deployed in safety-critical applications such as content moderation and video analytics. To process long videos efficiently, VideoLLMs rely on frame sampling, token compression, and modality fusion, which together form an observation pipeline that reduces the raw video to a compact internal representation. Recent observation-level attacks exploit this pipeline to prevent the model from perceiving harmful content, yet no defense has been explicitly designed for this threat. We introduce DefTEval, a controlled evaluation framework that systematically assesses whether input-level adversarial defenses, which operate on the pixel content of already-sampled frames, can mitigate observation-level attacks. Across five VideoLLMs, eleven representative defenses, and five attack types, we find that input-level defenses offer limited and inconsistent protection, with harmful detection rates frequently near zero. Critically, defenses fail even against attacks that embed harmful signals in every sampled frame, indicating that the bottleneck extends beyond sampling omission to the suppression of signals that do enter the model. Token compression discards localized features, and modality fusion systematically down-weights weakened visual signals. Furthermore, defense effectiveness is dominated by model architecture rather than by the defense method itself, and detection rates vary drastically across content categories, exposing structural weaknesses in temporal reasoning. These findings demonstrate that securing VideoLLMs requires system-level robustness mechanisms spanning sampling-aware coverage guarantees, token-level preservation of safety-relevant features, and modality-balanced fusion.