Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper critiques the conventional evaluation of certified training methods for neural networks, highlighting the misleading nature of reporting single configurations due to the inherent trade-off between natural and certified accuracy. By employing Pareto front comparisons and efficient automated multi-objective hyperparameter optimization, the authors identify a range of Pareto-optimal configurations that reveal significant undertuning in existing methods. The findings establish a new state of the art in certified training performance and provide a comprehensive multi-objective comparison that challenges previous assumptions about the effectiveness of various approaches.
Reporting a single configuration can obscure the true performance of certified training methods, as this study reveals substantial undertuning and establishes a new state of the art through Pareto front analysis.
Deep neural networks achieve strong performance on many supervised learning tasks but remain vulnerable to adversarial perturbations. Neural network verification provides mathematically rigorous robustness guarantees, yet at substantial computational cost. To mitigate this, certified training techniques optimise for verifiable robustness during training, typically inducing a trade-off between natural and certified accuracy controlled by method-specific hyperparameters. Because these metrics are inherently conflicting, the common practice of reporting a single configuration is problematic: it can mislead conclusions about overall performance and prevents unbiased assessments of the state of the art. We address this by evaluating certified training methods via Pareto front comparisons over the natural--certified accuracy trade-off. To enable fair, method-agnostic comparisons, we perform efficient automated multi-objective hyperparameter optimisation to identify a set of Pareto-optimal configurations for each method. This approach often uncovers substantial undertuning in previously reported configurations, yielding superior performance and establishing a new state of the art. Leveraging these fronts, we present the first comprehensive multi-objective comparison of certified training approaches, showing that prior advancements are less pronounced than assumed and revealing previously unreported performance complementarities.