Search papers, labs, and topics across Lattice.
The paper introduces "adversarial smuggling attacks" that exploit the gap between human and AI perception to bypass MLLM content moderation by encoding harmful content in human-readable, AI-unreadable visual formats. They categorize these attacks into Perceptual Blindness and Reasoning Blockade pathways and construct SmuggleBench, a benchmark of 1,700 instances, to evaluate the vulnerability of MLLMs. Experiments show that state-of-the-art MLLMs, including GPT-4V and Qwen3-VL, exhibit Attack Success Rates exceeding 90% on SmuggleBench, highlighting the urgent need for robust defenses.
MLLMs can be tricked into missing 90% of harmful content simply by encoding it in images that humans can easily read.
Multimodal Large Language Models (MLLMs) are increasingly being deployed as automated content moderators. Within this landscape, we uncover a critical threat: Adversarial Smuggling Attacks. Unlike adversarial perturbations (for misclassification) and adversarial jailbreaks (for harmful output generation), adversarial smuggling exploits the Human-AI capability gap. It encodes harmful content into human-readable visual formats that remain AI-unreadable, thereby evading automated detection and enabling the dissemination of harmful content. We classify smuggling attacks into two pathways: (1) Perceptual Blindness, disrupting text recognition; and (2) Reasoning Blockade, inhibiting semantic understanding despite successful text recognition. To evaluate this threat, we constructed SmuggleBench, the first comprehensive benchmark comprising 1,700 adversarial smuggling attack instances. Evaluations on SmuggleBench reveal that both proprietary (e.g., GPT-5) and open-source (e.g., Qwen3-VL) state-of-the-art models are vulnerable to this threat, producing Attack Success Rates (ASR) exceeding 90%. By analyzing the vulnerability through the lenses of perception and reasoning, we identify three root causes: the limited capabilities of vision encoders, the robustness gap in OCR, and the scarcity of domain-specific adversarial examples. We conduct a preliminary exploration of mitigation strategies, investigating the potential of test-time scaling (via CoT) and adversarial training (via SFT) to mitigate this threat. Our code is publicly available at https://github.com/zhihengli-casia/smugglebench.
UW