Search papers, labs, and topics across Lattice.
This paper introduces Calibrated Prediction-Powered Inference (CPPI), a method for semisupervised mean estimation that post-hoc calibrates a black-box prediction model's output on a labeled sample to improve its alignment with the outcome scale. CPPI uses either linear or isotonic calibration, and theoretical analysis demonstrates that isotonic calibration achieves first-order optimality, improving both predictive accuracy and estimator efficiency. Experiments show that CPPI often outperforms existing methods like PPI and is competitive with AIPW and PPI++.
Fixing miscalibrated black-box predictions with a simple post-hoc calibration step can significantly boost the accuracy and efficiency of semisupervised mean estimation.
We study semisupervised mean estimation with a small labeled sample, a large unlabeled sample, and a black-box prediction model whose output may be miscalibrated. A standard approach in this setting is augmented inverse-probability weighting (AIPW) [Robins et al., 1994], which protects against prediction-model misspecification but can be inefficient when the prediction score is poorly aligned with the outcome scale. We introduce Calibrated Prediction-Powered Inference, which post-hoc calibrates the prediction score on the labeled sample before using it for semisupervised estimation. This simple step requires no retraining and can improve the original score both as a predictor of the outcome and as a regression adjustment for semisupervised inference. We study both linear and isotonic calibration. For isotonic calibration, we establish first-order optimality guarantees: isotonic post-processing can improve predictive accuracy and estimator efficiency relative to the original score and simpler post-processing rules, while no further post-processing of the fitted isotonic score yields additional first-order gains. For linear calibration, we show first-order equivalence to PPI++. We also clarify the relationship among existing estimators, showing that the original PPI estimator is a special case of AIPW and can be inefficient when the prediction model is accurate, while PPI++ is AIPW with empirical efficiency maximization [Rubin et al., 2008]. In simulations and real-data experiments, our calibrated estimators often outperform PPI and are competitive with, or outperform, AIPW and PPI++. We provide an accompanying Python package, ppi_aipw, at https://larsvanderlaan.github.io/ppi-aipw/.
