Search papers, labs, and topics across Lattice.
This paper introduces a black-box auditing framework for Rényi differential privacy (RDP) in machine learning algorithms, using hypothesis testing to estimate Rényi divergence via Donsker-Varadhan (DV) estimators. They derive explicit, non-asymptotic confidence intervals for RDP auditing with class-restricted DV estimators, isolating statistical error from privacy leakage. The authors prove matching minimax lower bounds, demonstrating near-optimal sample complexity for auditing RDP via DV estimators, and empirically validate the framework on DP-SGD across MNIST and CIFAR-10, showing improved RDP lower bounds compared to existing methods.
You can now audit Rényi differential privacy with near-optimal sample complexity, thanks to a new framework that directly estimates Rényi divergence using Donsker-Varadhan estimators.
We study black-box auditing for machine learning algorithms that claim R \ 'enyi differential privacy (RDP) guarantees. We introduce an auditing framework, based on hypothesis testing, that directly estimates Rényi divergence between neighboring executions using the Donsker-Varadhan (DV) variational estimator. Our analysis yields explicit and non-asymptotic confidence intervals for RDP auditing via class-restricted DV estimators, separating statistical estimation error from algorithmic privacy leakage. We prove matching minimax lower bounds showing that, up to logarithmic factors, our sample-complexity guarantees are information-theoretically optimal, thereby establishing the first optimal guarantees for auditing RDP via DV estimators. Empirically, we instantiate our framework for auditing DP-SGD in a fully black-box setting. Across MNIST and CIFAR-10, and over a wide range of privacy regimes, our auditors produce a strong overall improvement on empirical RDP lower bounds compared to prior state-of-the-art black-box methods especially at small and moderate Rényi orders where accurate auditing is most challenging.
MIT CSAIL