Search papers, labs, and topics across Lattice.
This paper investigates the effectiveness of Bayesian optimization for identifying a strong single expert in large language models (LLMs) without the need for gradient-based optimization. By leveraging a random linear embedding of weight space and employing a Gaussian process surrogate, the authors demonstrate that their method can achieve superior performance with significantly fewer candidate evaluations compared to existing methods like RandOpt. The findings reveal that this approach not only reduces evaluation costs but also enhances the quality of the resulting models across various reasoning benchmarks.
Surrogate-guided Bayesian optimization can outperform traditional methods in finding strong single experts in LLMs while cutting evaluation costs by five times.
Gradient-free post-training has emerged as a compelling alternative to gradient-based optimization for large language models (LLMs), but existing approaches remain costly. We ask whether structured search can identify a strong single expert under a modest evaluation budget. Motivated by evidence that useful weight updates lie in low-dimensional subspaces, we apply Bayesian optimization within a random linear embedding of weight space. Our method requires no backpropagation and uses a Gaussian process surrogate to guide candidate evaluations efficiently. Across several reasoning benchmarks with Qwen2.5-Instruct models from 0.5B to 3B parameters, Bayesian optimization using five times less candidate evaluations matches or exceeds RandOpt. These results show that surrogate-guided search can substantially reduce the evaluation cost of gradient-free post-training while producing stronger deployable single experts.
