Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
The authors investigate the core assumption of scientific $\Delta$-learning—that baselines yielding smaller residual scales improve downstream performance—by evaluating molecular graph neural networks on quantum chemical total energy prediction. They find that complex local descriptor baselines produce small residuals with high landscape roughness that degrade model learnability, whereas semi-empirical baselines reduce both scale and roughness to boost in- and out-of-domain generalization. Consequently, they establish target space formulation as a primary design axis and provide scale-normalized graph Dirichlet roughness ($D_{\text{IQR}}$) as a pre-training metric for residual learnability.
Smaller residuals do not guarantee better scientific ML: complex physical baselines can create deceptively rough target landscapes that severely cripple downstream generalization.
In scientific machine learning, $\Delta$-learning trains models on residual errors relative to physical baselines, assuming that more accurate baselines with smaller residual scales inherently improve downstream performance. Here, we demonstrate that residual scale alone is an insufficient heuristic for learnability. Evaluating molecular graph neural networks on total energy targets, we show that complex local descriptor baselines can yield small residual targets that are disproportionately rough within architecture-informed proxy spaces and harder to learn relative to their scale. Conversely, semi-empirical baseline reduces both scale and normalized roughness, improving in-domain and out-of-domain prediction. We introduce scale-normalized graph Dirichlet roughness ($D_{\text{IQR}}$) as a pre-training diagnostic for residual learnability and establish baseline complementarity as a core target-design principle, elevating target space formulation alongside model architecture as a key axis for scientific machine learning.