Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the inconsistent performance of chemical language models (CLMs) on molecular property prediction (MPP) tasks by conducting controlled experiments varying dataset size, model size, and standardization techniques. The study reveals the impact of these factors on both pre-training and fine-tuning performance, providing numerical evidence to clarify the behavior of CLMs in MPP. The findings highlight overlooked aspects influencing CLM performance, offering insights for improved model design and application in chemical property prediction.
Chemical language models' inconsistent performance on molecular property prediction tasks isn't just random noise – it's systematically tied to dataset size, model size, and standardization, factors often overlooked in existing literature.
Chemical language models (CLMs) have emerged as promising competitors to popular classical machine learning models for molecular property prediction (MPP) tasks. However, an increasing number of studies have reported inconsistent and contradictory results for the performance of CLMs across various MPP benchmark tasks. In this study, we conduct and analyze hundreds of meticulously controlled experiments to systematically investigate the effects of various factors, such as dataset size, model size, and standardization, on the pre-training and fine-tuning performance of CLMs for MPP. In the absence of well-established scaling laws for encoder-only masked language models, our aim is to provide comprehensive numerical evidence and a deeper understanding of the underlying mechanisms affecting the performance of CLMs for MPP tasks, some of which appear to be entirely overlooked in the literature.