Search papers, labs, and topics across Lattice.
This paper introduces a unified membership inference framework that effectively addresses privacy risks posed by generative models across text-to-text, text-to-image, and image-to-text modalities. By leveraging the observation that the output distribution of generative models can approximate their training data distribution, the authors employ likelihood ratio testing to infer membership in a shared embedding space. Experimental results reveal that this cross-modal approach outperforms existing state-of-the-art methods, which are limited to individual model classes, highlighting its practical utility in real-world scenarios.
Membership inference attacks can now be effectively executed across multiple generative modalities with a single, unified framework.
Large generative models across text-to-text, text-to-image, and image-to-text modalities have been shown to pose significant privacy risks. One fundamental threat is membership inference attacks (MIA), which aim to determine whether a given data point was used in a model's training set. Although prior work has investigated MIAs against these three classes of generative models, existing approaches treat them in isolation and are not cross-applicable, thereby limiting their real-world utility. To address this limitation, we present the first comprehensive study of a unified membership inference framework that applies across text-to-text, text-to-image, and image-to-text modalities. Our approach is grounded in a key modality-agnostic observation: the output distribution of a generative model can approximate its training data distribution. Leveraging this property, we model the distributions of model-generated outputs and auxiliary non-member samples in a shared embedding space, and perform membership inference via likelihood ratio testing. We conduct extensive experiments in a strict black-box setting under both partial-knowledge and zero-knowledge threat models, and evaluate membership inference against both fine-tuning and pre-training data. Experimental results demonstrate our approach's superior performance in comparison to existing state-of-the-art methods, which are typically optimized for a single model class.
