Search papers, labs, and topics across Lattice.
The paper addresses practical challenges in training deep heteroskedastic regression models, specifically the trade-off between uncertainty quantification and mean prediction. They identify fallacies in existing approaches and propose a post-hoc method that fits a variance model across intermediate layers of a pre-trained network using a hold-out dataset. The proposed method achieves state-of-the-art uncertainty quantification on molecular graph datasets while maintaining mean prediction accuracy and computational efficiency.
Unlock accurate uncertainty estimates in your pre-trained regression models without retraining the whole network, using a simple post-hoc variance fitting trick.
Uncertainty quantification (UQ) in deep learning regression is of wide interest, as it supports critical applications including sequential decision making and risk-sensitive tasks. In heteroskedastic regression, where the uncertainty of the target depends on the input, a common approach is to train a neural network that parameterizes the mean and the variance of the predictive distribution. Still, training deep heteroskedastic regression models poses practical challenges in the trade-off between uncertainty quantification and mean prediction, such as optimization difficulties, representation collapse, and variance overfitting. In this work we identify previously undiscussed fallacies and propose a simple and efficient procedure that addresses these challenges jointly by post-hoc fitting a variance model across the intermediate layers of a pretrained network on a hold-out dataset. We demonstrate that our method achieves on-par or state-of-the-art uncertainty quantification on several molecular graph datasets, without compromising mean prediction accuracy and remaining cheap to use at prediction time.
