Search papers, labs, and topics across Lattice.
This paper introduces Prompt-Optimized Parameter Shaking (POPS), an adversarial strategy that exploits the weaknesses of Multi-modality Machine Unlearning (MMU) in Multimodal Large Language Models (MLLMs) to recover supposedly unlearned private information. By optimizing prompt suffixes, POPS generates outputs that allow for fine-tuning the models, effectively revealing sensitive data that was intended to be erased. The experiments demonstrate that existing MMU methods are significantly flawed, with POPS achieving near-complete recovery of sensitive information, highlighting critical vulnerabilities in current privacy protections for MLLMs.
Existing Multi-modality Machine Unlearning methods are fundamentally flawed, allowing adversaries to recover nearly all supposedly erased sensitive information from MLLMs.
Multimodal Large Language Models (MLLMs) have demonstrated impressive performance on cross-modal tasks by jointly training on large-scale textual and visual data, where privacy-sensitive examples could be unintentionally encoded, raising concerns about privacy or copyright violation. To this end, Multi-modality Machine Unlearning (MMU) was proposed as a mitigation that can effectively force MLLMs to forget private information. However, the robustness of such unlearning methods is not fully exploited when the model is published and accessible to malicious users. In this paper, we propose a novel adversarial strategy, namely Prompt-Optimized Parameter Shaking (POPS), aiming to recover the supposedly unlearned multi-modality knowledge from the MLLMs. Our method elicits the victim MLLMs to generate potential private examples via prompt-suffix optimization, and then exploits these synthesized outputs to fine-tune the models so they disclose the true private information. The experiments on the different MMU benchmarks reveal substantial weaknesses in the existing MMU algorithms. Our POPS can even achieve a near-complete recovery of supposedly erased sensitive information on the unlearned MLLMs, exposing fundamental vulnerabilities that challenge the foundational robustness of representative MMU-based privacy protections.
