Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Online Hyperparameter Optimization (OHPO) as a formal framework for optimizing hyperparameters in production systems using live feedback, addressing the challenge of noisy data without representative validation. The authors propose IMABO, a versatile framework that integrates various bandit policies with oracles for generating new configurations, and demonstrate its effectiveness through the instantiation of IMOSS, which achieves a proven expected cumulative quantile-regret bound. Experimental results show that IMABO significantly outperforms traditional methods across multiple settings, including classical machine learning and LLM-based agents, by minimizing cumulative regret.
IMABO achieves the lowest cumulative regret in hyperparameter optimization for production systems, outperforming traditional methods by effectively leveraging live feedback.
Many production systems can assess a configuration only by using it on live requests and observing noisy feedback. Modern agentic systems are a prominent example, with inference-time choices such as model selection, retrieval depth, prompting strategy, and decoding temperature, yet often with no representative validation data. We formalize this setting as Online Hyperparameter Optimization (OHPO) and cast it as an infinitely many-armed bandit over mixed and conditional search spaces. We introduce IMABO, a general framework that combines any bandit policy for choosing among already sampled configurations with any oracle for proposing new ones. We instantiate it with IMOSS, a restart-free anytime policy whose active set grows as $t^β$, and prove an expected cumulative quantile-regret bound of $O(p_ρ^{-1/β} + T^{(1+β)/2})$, where $β\in(0,1)$ controls active-set growth and $p_ρ$ lower-bounds the probability that a proposed configuration falls in the top-$ρ$ fraction of the search space. We combine IMOSS with three practical oracles: a Tree-structured Parzen Estimator, an incumbent-mutation oracle driven by a per-coordinate bandit, and a pretrained tabular foundation model, all three improving over the uniform random oracle baseline. IMABO obtains the lowest cumulative regret across diverse OHPO settings, from tuning classical machine-learning models to configuring LLM-based agents.