Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper explores the integration of latent-space Bayesian optimization (LSBO) with tabular foundation models to enhance their performance in structured object design, particularly for molecules and proteins. By addressing the mismatch between the pretraining tasks of these models and the unique requirements of LSBO, the authors introduce a continued-pretraining approach that incorporates synthetic optimization tasks in the latent space of a molecular VAE. The resulting model demonstrates significant improvements on held-out molecular optimization benchmarks, highlighting the effectiveness of LSBO-specific adaptations for in-context learning surrogates.
Addressing the mismatch in pretraining tasks, this approach significantly boosts the performance of tabular foundation models in molecular optimization tasks.
Bayesian optimization (BO) is a central tool for sample-efficient design, and latent-space Bayesian optimization (LSBO) extends it to structured objects such as molecules and proteins. In parallel, tabular foundation models such as TabPFN and TabICL now achieve state-of-the-art regression performance and are increasingly used as BO surrogates. Because their Bayesian behavior is induced by large synthetic pretraining collections, the composition of this pretraining distribution is crucial. LSBO creates a distinctive mismatch: the induced map from latent code to objective value differs markedly from the regression tasks used to train current in-context models. We address this mismatch by complementing the pretraining stage of tabular foundation model surrogates with synthetic optimization tasks defined on the latent space of a molecular VAE. The continued-pretraining objective features a regularizer that anchors the model to the original checkpoint, preserving its broad regression prior while avoiding overspecialization to the adaptation tasks. On held-out molecular optimization benchmarks, the resulting model achieves strong performance, supporting the relevance of LSBO-specific adaptation for in-context surrogates.