Search papers, labs, and topics across Lattice.
This paper addresses the challenge of distinguishing between training data and generated outputs in generative models, formalized as Member vs Generated Inference (MGI). The authors identify that existing membership inference and attribution methods fail due to their reliance on likelihood signals that do not adequately differentiate between true training members and generated samples. To overcome these limitations, they introduce the Data Circuit Breaker (DCB), a three-stage approach that effectively utilizes signals from both an autoencoder and a latent generator, demonstrating consistent success across various generative models, even in complex scenarios involving near-duplicates and model derivatives.
Existing methods misclassify generated samples as training members, but the Data Circuit Breaker reliably distinguishes between the two, even in challenging contexts.
As generative models increasingly produce samples that are indistinguishable from human-created content, it becomes difficult to determine whether a given data point was part of a model's natural training set or was generated by the model itself, especially when models memorize and reproduce training data. We formalize this challenge as Member vs Generated Inference (MGI): given a sample and a target generative model, infer whether the sample is a true training member or a generated output of that model. Focusing on image generation, we show that existing membership inference methods systematically misclassify generated samples as training members, while attribution-based methods often misclassify true members as generated. This failure arises because both approaches rely on likelihood-related signals that are similarly elevated for training examples and for the model's own outputs. To address MGI, we propose Data Circuit Breaker (DCB), a three-stage method that combines complementary signals from a generative model's autoencoder and latent generator to distinguish training members from generated samples. Across multiple generative models, including image autoregressive and diffusion models, DCB consistently addresses the shortcomings of membership inference and attribution methods, remains effective even when models reproduce near-duplicates of training samples, and generalizes to challenging model derivative settings in which new models are trained on generated data.
