Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces an E-value based stopping rule for Bayesian Deep Ensembles (BDEs) to determine when MCMC sampling no longer significantly improves upon the initial Deep Ensemble (DE) baseline. The ensemble construction is framed as a sequential anytime-valid hypothesis test, allowing for early stopping of the sampling process when the null hypothesis (MCMC offers no improvement) cannot be rejected. Experiments across diverse settings demonstrate that the proposed approach can often achieve comparable performance to full-chain sampling with only a fraction of the computational budget.
Stop wasting compute on MCMC sampling for Bayesian Deep Ensembles – this E-value based stopping rule shows you when you've extracted all the benefit you can get.
Bayesian Deep Ensembles (BDEs) represent a powerful approach for uncertainty quantification in deep learning, combining the robustness of Deep Ensembles (DEs) with flexible multi-chain MCMC. While DEs are affordable in most deep learning settings, (long) sampling of Bayesian neural networks can be prohibitively costly. Yet, adding sampling after optimizing the DEs has been shown to yield significant improvements. This leaves a critical practical question: How long should the sequential sampling process continue to yield significant improvements over the initial optimized DE baseline? To tackle this question, we propose a stopping rule based on E-values. We formulate the ensemble construction as a sequential anytime-valid hypothesis test, providing a principled way to decide whether or not to reject the null hypothesis that MCMC offers no improvement over a strong baseline, to early stop the sampling. Empirically, we study this approach for diverse settings. Our results demonstrate the efficacy of our approach and reveal that only a fraction of the full-chain budget is often required.