Track trials, findings, and clinical evidence as it lands.
Search papers, labs, and topics across Lattice.
This paper introduces a bilevel Stackelberg formulation to jointly train neural networks for both accuracy and calibration end-to-end, eliminating the need for separate calibration datasets and optimization steps. By integrating calibration directly into the backbone training via cross-entropy, the approach achieves state-of-the-art calibration performance with minimal added complexity, even when using the final classification layer as the calibrator. This method offers a more efficient and integrated solution compared to traditional post-hoc calibration techniques, particularly in complex scenarios like multi-task learning.
End-to-end calibration is achievable by treating the calibrator as a bilevel optimization problem, yielding accurate and well-calibrated models without extra parameters or datasets.
Neural networks (NNs), while highly flexible and versatile, are often poorly calibrated. Post-hoc calibration methods adjust the outputs of an already trained model using a separate calibration dataset and additional calibration parameters. This process adds overhead that requires separate optimization of the calibrator and subsequent model selection. In this paper, we show that standard (post-hoc) calibrators can be trained end-to-end with the backbone via a bilevel Stackelberg formulation. This formulation eliminates the need for a separate calibration set or a dedicated calibration loss by jointly optimizing accuracy and calibration using cross-entropy, introducing only one additional hyperparameter: the calibrator learning rate. Although our framework supports calibrators ranging from temperature and matrix scaling to NN layers, our results show that using the final classification layer itself as the calibrator yields accurate, well-calibrated models with no additional parameters. Our approach outperforms SOTA calibrators with minimal added complexity, while also avoiding the multiplicative overhead of separate calibration components in settings such as multi-task learning.