Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Large language Model Bayesian Optimization (LlMBO), a gradient-free global optimization approach for fine-tuning LLMs with very few parameters, inspired by TinyLoRA's low-parameter strategy. LlMBO is shown to be sample-efficient, memory-light, and capable of direct optimization of greedy-decoding accuracy, unlike gradient-based methods. Experiments demonstrate LlMBO matches or surpasses GRPO's fine-tuning gains on a math benchmark suite while requiring significantly less compute and memory.
Global optimization unlocks LLM fine-tuning with drastically reduced compute and memory, matching or exceeding gradient-based gains.
Recently, Morris et al. introduced TinyLoRA, a strategy for LLM fine-tuning which introduces arbitrarily few parameters, and demonstrated accuracy improvements using less than $10$ parameters rather than the ${\gtrsim 10,000}$ typically demanded by LoRA-based methods. However, that study continued to use gradient-based local search methods of the kind deployed for training multi-billion parameter LLMs in the first place. This article is predicated on the observation that, in the low dimensional setting, global optimization methods become tractable, unlocking Large language Model Bayesian Optimization (LlMBO). LlMBO is sample-efficient, gradient-free, memory-light, parallelizes across devices without synchronization, and, unlike policy-gradient methods, can optimize greedy-decoding accuracy directly. We describe numerical experiments comparing LlMBO against GRPO on fine-tuning tasks for a variety of models on a math benchmark suite which show that LlMBO can be a suitable alternative in this regime, matching or exceeding GRPO's fine tuning gains, while using a fraction of the compute and memory.