Search papers, labs, and topics across Lattice.
This paper introduces a dual-objective surrogate-guided ensemble search method that optimizes both the predictive accuracy and diversity of neural architectures represented as directed acyclic graphs. By training two independent surrogate models to estimate these metrics, the approach effectively navigates the computationally intractable search space associated with neural ensemble search (NES). The resulting ensemble outperforms standard baselines like Deep Ensembles and Random Search across multiple datasets, demonstrating the method's efficacy in enhancing model robustness and performance.
Surrogate-guided ensemble search can identify architectures that are not only strong individually but also diverse, outperforming traditional methods on benchmark datasets.
Ensembles are a standard way to improve the performance and robustness of deep neural networks, but their effectiveness crucially depends on both the quality and the diversity of individual models. Most neural architecture search (NAS) methods are computationally expensive. Extending them to neural ensemble search (NES), which requires joint optimization of individual architectures and their ensemble composition, leads to an exponential growth of the search space and makes the problem computationally intractable. To address this, we introduce a dual-objective surrogate-guided ensemble search: candidate architectures are represented as directed acyclic graphs, and two surrogate models are trained independently to estimate predictive accuracy and diversity potential. Their combined estimates guide an NES framework that efficiently identifies architectures that are both individually strong and collectively diverse. Our final ensemble achieves competitive or superior performance compared to standard baselines such as Deep Ensembles and Random Search on FashionMNIST, CIFAR-10, and CIFAR-100.
