Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel canary crafting method for one-run privacy auditing that optimizes for both high detectability and minimal interference between canaries. They combine influence function-based initialization with bilevel optimization to maximize canary distinguishability while promoting diversity in the embedding space. Experiments demonstrate that this approach achieves stronger privacy leakage estimates with lower computational cost compared to existing canary crafting methods.
Forget expensive multi-run privacy audits: this canary crafting method lets you estimate privacy leakage more accurately and cheaply in a single training run.
Privacy auditing aims to empirically assess privacy leakage in machine learning models using membership inference attacks (MIAs), and to derive lower bounds on differential privacy (DP) parameters. Recent one-run auditing methods address the high cost of standard approaches by relying on a single training run with multiple "canary" points whose inclusion or exclusion must be detected by the auditor. In this work, we study the problem of efficiently crafting canaries for one-run privacy auditing. Motivated by recent theoretical insights suggesting that interference between canaries contributes to weaker leakage estimates compared to multi-run methods, we propose to optimize canaries to be both highly detectable and minimally interfering. Our approach combines a greedy initialization based on influence functions with a bilevel optimization procedure that maximizes distinguishability while promoting diversity in embedding space, enabling the use of computationally efficient bilevel algorithms. Experiments show that our method achieves stronger privacy leakage estimates at a lower computational cost than existing canary crafting approaches.