Search papers, labs, and topics across Lattice.
This paper investigates the privacy vulnerabilities of Multi-modal Large Language Models (MLLMs), particularly their ability to leak sensitive information from images, a risk not fully addressed in text-only models. By introducing the MM-Privacy dataset, the authors evaluate various MLLMs for Disclosure and Retention Risks, revealing significant data leakage across multiple tasks. The findings underscore the critical need for effective mitigation strategies to safeguard sensitive information in MLLMs, highlighting the unique privacy challenges they present.
MLLMs can leak sensitive information from images, exposing new privacy risks that traditional models do not face.
Privacy risks in text-only Large Language Models (LLMs) are well studied, particularly their tendency to memorize and leak sensitive information. However, Multi-modal Large Language Models (MLLMs), which process both text and images, introduce unique privacy challenges that remain underexplored. Compared to text-only models, MLLMs can extract and expose sensitive information embedded in images, posing new privacy risks. We reveal that some MLLMs are susceptible to privacy breaches, leaking sensitive data embedded in images or stored in memory. Specifically, in this paper, we (1) introduce MM-Privacy, a comprehensive dataset designed to assess privacy risks across various multi-modal tasks and scenarios, where we define Disclosure Risks and Retention Risks. (2) systematically evaluate different MLLMs using MM-Privacy and demonstrate how models leak sensitive data across various tasks, and (3) provide additional insights into the role of task inconsistency in privacy risks, emphasizing the urgent need for mitigation strategies. Our findings highlight privacy concerns in MLLMs, underscoring the necessity of safeguards to prevent data exposure. Our dataset and code can be found here.
