Search papers, labs, and topics across Lattice.
This paper introduces AD-CERT, a novel certified training objective that integrates adversarial distillation with Interval Bound Propagation (IBP) upper bounds to enhance the certified robustness of neural networks. By distilling adversarial information from a robust teacher model in the logit space, AD-CERT achieves superior certified performance while maintaining standard accuracy, addressing the trade-off challenges faced by existing methods. The results demonstrate a significant improvement in certified accuracy, achieving up to 5.40 percentage points over previous robust feature-space distillation approaches across multiple benchmarks.
Adopting adversarial distillation can boost certified accuracy by over 5% while maintaining robust performance, reshaping the landscape of certified training.
Certified training aims to produce models whose predictions can be formally verified against adversarial perturbations, typically by optimising upper bounds on the worst-case loss over an allowed perturbation set. For neural networks, certified training methods based purely on tight relaxation bounds produce networks that are amenable to certification, but sacrifice standard accuracy. Conversely, adversarial training often yields stronger empirical robustness and standard accuracy, but the resulting models are generally difficult to certify with neural network verifiers. Recently, the literature has shown that better standard-certified accuracy trade-offs can be achieved by combining adversarial training objectives with loose over-approximations based on Interval Bound Propagation (IBP), effectively interpolating between lower and upper bounds of the worst-case loss. Building on this, we introduce AD-CERT, a certified training objective that combines adversarial distillation with an IBP upper bound. We show that distilling adversarial information over the logit space from an empirically robust teacher provides an effective lower bound surrogate for certified training, with AD-CERT achieving state-of-the-art certified performance on several robustness benchmarks. Furthermore, in a unified setup, distilling adversarial information at the logit-level is shown to improve certified accuracy over a robust feature-space distillation objective by up to 5.40 percentage points.
