Search papers, labs, and topics across Lattice.
This paper introduces SeMI*, an "optimal" Membership Inference (MI) attack that leverages a sequence of model updates to detect the presence of a target data point inserted at a specific update step. They derive the optimal power of SeMI* for empirical mean computation with finite samples, with or without privacy, and show that accessing the model sequence avoids the dilution of MI signals seen in attacks on the final model. Experiments on models trained with DP-SGD demonstrate that practical variants of SeMI* lead to tighter privacy audits compared to existing baselines.
Knowing the sequence of model updates lets you pinpoint when a specific data point was added to the training set, making membership inference attacks far more potent than previously thought.
Modern AI models are not static. They go through multiple updates in their lifecycles. Thus, exploiting the model dynamics to create stronger Membership Inference (MI) attacks and tighter privacy audits are timely questions. Though the literature empirically shows that using a sequence of model updates can increase the power of MI attacks, rigorous analysis of the `optimal'MI attacks is limited to static models with infinite samples. Hence, we develop an `optimal'MI attack, SeMI*, that uses the sequence of model updates to identify the presence of a target inserted at a certain update step. For the empirical mean computation, we derive the optimal power of SeMI*, while accessing a finite number of samples with or without privacy. Our results retrieve the existing asymptotic analysis. We observe that having access to the model sequence avoids the dilution of MI signals unlike the existing attacks on the final model, where the MI signal vanishes as training data accumulates. Furthermore, an adversary can use SeMI* to tune both the insertion time and the canary to yield tighter privacy audits. Finally, we conduct experiments across data distributions and models trained or fine-tuned with DP-SGD demonstrating that practical variants of SeMI* lead to tighter privacy audits than the baselines.
