Search papers, labs, and topics across Lattice.
This paper introduces KENDO, a unified framework that enhances Bayesian optimization (BO) and active learning (AL) by integrating Ensemble Gaussian Processes with disagreement-aware acquisition strategies. By replacing traditional hyperparameter sampling with a kernel ensemble and adaptive Bayesian weighting, KENDO demonstrates significant improvements in both optimization performance and computational efficiency. Experimental results show that KENDO-BO outperforms state-of-the-art methods while reducing computational overhead by up to 5 times, and KENDO-AL achieves superior predictive calibration with a speedup of up to 27 times compared to MCMC-based approaches.
KENDO achieves up to 5x faster Bayesian optimization and 27x faster active learning while enhancing predictive performance.
Hyperparameter selection remains a key challenge in Bayesian optimization (BO) and Bayesian active learning (AL), as model misspecification can lead to suboptimal performance, while more accurate fully Bayesian treatments typically rely on computationally expensive MCMC sampling. This paper proposes a unified framework, KENDO (Kernel ENsemble Disagreement-aware Operator), that integrates Ensemble Gaussian Processes (EGP) with disagreement-aware acquisition strategies. The central idea is to replace hyperparameter sampling with a kernel ensemble and adaptive Bayesian weighting, combined with disagreement-aware acquisition strategies. Within this unified framework, we instantiate KENDO-BO for BO and KENDO-AL for Bayesian AL, demonstrating that both arise from a common self-correcting mechanism with task-specific acquisition objectives. We further extend the approach to multi-objective optimization via random scalarization that preserves the single-optimizer conditioning structure. Thorough numerical tests on synthetic and real-world benchmarks across single-objective optimization, multi-objective optimization, and active learning demonstrate that (i) KENDO-BO achieves competitive or superior optimization performance compared to state-of-the-art methods while reducing computational overhead by up to $5\times$ and (ii) KENDO-AL achieves superior predictive calibration over MCMC-based active learning baselines with up to $27\times$ speedup.
