Search papers, labs, and topics across Lattice.
This paper unifies various black-box optimization methods, including Evolution Strategies (ES), Consensus-Based Optimization (CBO), and Optimization via Integration (OVI), under a common theoretical framework that highlights their differences in fitness aggregation and consensus scope. By introducing hybrid optimizers that blend these approaches, the authors enable explicit control over the preference for flat minima, enhancing performance and robustness in continuous control tasks. The proposed hybrids achieve competitive results in language model merging and outperform traditional methods on standard benchmarks and higher-dimensional locomotion tasks.
Hybrid optimizers that blend black-box methods can significantly enhance performance and robustness in optimization tasks, outperforming traditional approaches.
When gradient information is unavailable, black-box optimization (BBO) methods provide a practical alternative. While Evolution Strategies (ES), Consensus-Based Optimization (CBO), Optimization via Integration (OVI), and related methods have each been studied independently, their connections remain underexplored. We unify these approaches within a common theoretical framework, revealing that they differ primarily in two design choices: fitness aggregation (controlling sharpness preference) and consensus scope (controlling modality). Leveraging these insights, we introduce hybrid optimizers that interpolate between existing methods. Our ES-OVI hybrid allows explicit control over the preference for flat minima, enabling a trade-off between performance and robustness in continuous control tasks. Our CBO-OVI hybrids combine the higher-dimensional efficiency of parametric methods with the multimodal capabilities of particle-based approaches, achieving competitive results on language model merging under limited evaluation budgets. We validate our methods on standard BBO benchmarks and higher-dimensional locomotion tasks, demonstrating that the hybrid methods can outperform their constituent algorithms.
