Search papers, labs, and topics across Lattice.
This paper addresses the critical issue of confidence calibration in deep learning systems, particularly under conditions of label noise and domain shifts, which are common in high-stakes applications. The authors propose a novel framework that utilizes an estimated noise model to derive noise-free confidence estimates and extend this to a noise-aware Conformal Prediction method that ensures reliable uncertainty quantification. Additionally, they introduce a locally differentially private conformal prediction framework, enabling valid predictions while safeguarding user privacy, thereby enhancing the practical deployment of neural networks in safety-critical environments.
Noise-aware calibration methods can transform unreliable predictions into trustworthy confidence estimates, even in privacy-preserving contexts.
In high-stakes applications, reliable confidence estimates are as important as the predictions themselves. Confidence calibration ensures that predicted probabilities reflect the likelihood of correctness, making it essential for safe deployment of deep learning models. However, existing methods typically assume access to clean validation data, which is often unrealistic due to label noise and domain shifts. This thesis develops methods for improving calibration under these conditions. First, we address calibration under label noise. Standard methods can produce misleading confidence estimates when labels are unreliable. We propose a framework that uses an estimated noise model to reconstruct noise-free confidence estimates by modeling the relationship between noisy and clean label distributions. We extend this approach to Conformal Prediction (CP), which provides set-valued predictions with guaranteed coverage. Our noise-aware CP method estimates clean conformity scores despite label noise, enabling reliable uncertainty quantification. Next, we study calibration in unsupervised domain adaptation, where a model trained on a labeled source domain is adapted to an unlabeled target domain. Since labeled target data are unavailable, we estimate target-domain accuracy from source performance and domain discrepancies, enabling calibration without target labels. We also consider privacy-preserving settings in which user labels and model outputs must remain protected. We propose a locally differentially private conformal prediction framework that provides valid uncertainty quantification while maintaining privacy guarantees and balancing privacy, computational feasibility, and prediction reliability. Our results bridge calibration theory and practical deployment in safety-critical applications, contributing to reliable, privacy-preserving, and noise-resilient neural network predictions.
