Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a jailbreak framework for vision-language models (VLMs) that combines Chain-of-Thought (CoT) prompting with a ReAct-driven adaptive noising mechanism to bypass safety filters. The adaptive noising iteratively perturbs input images based on model feedback, focusing on regions that trigger safety defenses. Experiments show that this dual-strategy significantly improves attack success rates (ASR) while preserving the naturalness of both text and visual inputs.
VLMs can be jailbroken with stealthy prompts by combining chain-of-thought reasoning with adaptive image noising that targets safety defenses.
Vision-language models (VLMs) have become central to tasks such as visual question answering, image captioning, and text-to-image generation. However, their outputs are highly sensitive to prompt variations, which can reveal vulnerabilities in safety alignment. In this work, we present a jailbreak framework that exploits post-training Chain-of-Thought (CoT) prompting to construct stealthy prompts capable of bypassing safety filters. To further increase attack success rates (ASR), we propose a ReAct-driven adaptive noising mechanism that iteratively perturbs input images based on model feedback. This approach leverages the ReAct paradigm to refine adversarial noise in regions most likely to activate safety defenses, thereby enhancing stealth and evasion. Experimental results demonstrate that the proposed dual-strategy significantly improves ASR while maintaining naturalness in both text and visual domains.