Search papers, labs, and topics across Lattice.
The paper introduces Visual Memory Injection (VMI), a novel attack that manipulates large vision-language models (LVLMs) in multi-turn conversations by injecting adversarial information through subtly perturbed images. VMI allows attackers to control the LVLM's output only when a specific triggering prompt is given, enabling targeted manipulation after extended interaction. Experiments on open-weight LVLMs demonstrate the feasibility of large-scale user manipulation via perturbed images in multi-turn settings, highlighting a significant vulnerability.
LVLMs can be subtly backdoored with manipulated images, allowing attackers to inject targeted messages into multi-turn conversations and manipulate users after a specific trigger.
Generative large vision-language models (LVLMs) have recently achieved impressive performance gains, and their user base is growing rapidly. However, the security of LVLMs, in particular in a long-context multi-turn setting, is largely underexplored. In this paper, we consider the realistic scenario in which an attacker uploads a manipulated image to the web/social media. A benign user downloads this image and uses it as input to the LVLM. Our novel stealthy Visual Memory Injection (VMI) attack is designed such that on normal prompts the LVLM exhibits nominal behavior, but once the user gives a triggering prompt, the LVLM outputs a specific prescribed target message to manipulate the user, e.g. for adversarial marketing or political persuasion. Compared to previous work that focused on single-turn attacks, VMI is effective even after a long multi-turn conversation with the user. We demonstrate our attack on several recent open-weight LVLMs. This article thereby shows that large-scale manipulation of users is feasible with perturbed images in multi-turn conversation settings, calling for better robustness of LVLMs against these attacks. We release the source code at https://github.com/chs20/visual-memory-injection
