Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to deriving non-vacuous generalization bounds for deep learning models by certifying the disagreement between a target model and a surrogate model with known generalization properties. The true risk of the target model is bounded by combining the surrogate's risk bound with a disagreement certificate estimated on an unlabeled dataset. The authors demonstrate the effectiveness of this approach using sample compression, model compression, and PAC-Bayes theory to train the surrogate models, achieving tight certificates without modifying the target model or its training procedure.
Get generalization guarantees for your deep learning model without changing its architecture or training procedure by bounding its risk against a well-understood surrogate.
Generalization bounds for deep learning models are typically vacuous, not computable or restricted to specific model classes. In this paper, we tackle these issues by providing new disagreement-based certificates for the gap between the true risk of any two predictors. We then bound the true risk of the predictor of interest via a surrogate model that enjoys tight generalization guarantees, and evaluating our disagreement bound on an unlabeled dataset. We empirically demonstrate the tightness of the obtained certificates and showcase the versatility of the approach by training surrogate models leveraging three different frameworks: sample compression, model compression and PAC-Bayes theory. Importantly, such guarantees are achieved without modifying the target model, nor adapting the training procedure to the generalization framework.
