Search papers, labs, and topics across Lattice.
This paper introduces a boundary-targeted membership inference attack (MIA) strategy that identifies low-confidence examples to infer the membership of training data used for safety classifiers. The approach exploits the hypothesis that examples near the decision boundary reveal memorization by the classifier, thus amplifying the signal of membership. Experiments demonstrate a significant improvement in recovering sensitive conversations related to user distress compared to existing MIA methods, highlighting vulnerabilities in safety classifiers.
Safety classifiers leak surprisingly sensitive information: a boundary-targeted attack recovers 19% of user distress conversations from the training data, far exceeding existing membership inference methods.
Safety classifiers are essential safeguards within generative AI systems, filtering harmful content or identifying at-risk users when interacting with large language models. Despite their necessity, these models are trained on sensitive datasets including discussions of self-harm and mental health, raising important, yet poorly understood, privacy concerns. Membership inference attacks (MIAs) allow adversaries to infer membership of examples used to train models. In this work, we hypothesize that identifying the examples on which the classifier is least confident are informative for an adversary to infer membership. This reflects a localized failure of generalization, where the model relies on memorization to resolve ambiguity in the training set. To investigate this, we introduce a new boundary-targeted selection strategy that identifies low confidence examples that amplify the signal of an examples membership within a training set. Our experimental results show that an adversary can recover 19\% of the conversations a safety classifier flagged as indicating user distress, at a 5\% false-positive rate, on a classifier fine-tuned for detecting a user who may require emotional support. This is $3.5$ times more than attacking using state-of-the-art MIA methods alone. Finally, we characterize the boundary laying examples and show that content-based filtering is ineffective for protection, and existing noise strategies can effectively mitigate susceptibility of these examples.
CMU ML