Search papers, labs, and topics across Lattice.
This study explores the impact of various matrix-structured optimizers, specifically SOAP, Muon, and the hybrid SOAP-Muon, on the training of machine learning interatomic potentials (MLIPs) like NequIP and Allegro. The findings reveal that these optimizers significantly enhance both convergence speed and final accuracy compared to the traditional Adam optimizer, with SOAP and SOAP-Muon consistently outperforming it, particularly under partial force supervision. This research highlights the critical yet often neglected role of optimizer selection in the design of MLIPs, suggesting that it can lead to substantial improvements in training efficiency and model performance.
Switching from Adam to SOAP or SOAP-Muon can dramatically accelerate training and boost accuracy in machine learning interatomic potentials.
Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community. Here, we implement and systematically compare a class of recently proposed matrix-structured optimizers, including Muon, SOAP, and the hybrid SOAP-Muon, for training NequIP and Allegro MLIP models. We find that these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon only provides partial gains relative to Adam. The improvements are particularly pronounced under partial force supervision. Our results indicate that optimizer choice is an overlooked yet impactful design axis for MLIPs.
