Search papers, labs, and topics across Lattice.
This paper explores the impact of exploiting both exact and approximate label symmetries on the data scaling behavior of machine learning models. The authors demonstrate that incorporating these symmetries, particularly in the context of modeling electron densities and potential energy surfaces of molecules, leads to improved learning curves and generalization efficiency. They further introduce a Hessian-based correction to mitigate errors arising from approximate symmetries, enhancing model performance.
Exploiting approximate label symmetries can significantly improve data scaling in machine learning, even when those symmetries aren't perfect.
Enforcing universal symmetries in machine learning (ML) models is a common strategy to mitigate data scarcity. We show that exploiting exact, as well as approximate, label symmetries can benefit scaling laws. We illustrate the idea for the s, p, d orbital densities of the electron in the hydrogen atom, for the three vibrational normal modes of the water molecule, as well as its full 3D potential energy hypersurface. Resulting ML models of electron density and potential energies exhibit superior learning curves, demonstrating improved generalization efficiency. When label symmetries are not exact, the same principles govern the observed learning behavior -- up to the convergence floors set by the degree to which the symmetry is approximate. For convex wells in the molecular potential energy surface, a Hessian-based correction suppresses the leading symmetry-breaking error in augmented labels.
Mila