Search papers, labs, and topics across Lattice.
This paper introduces Prediction-Powered Active Testing (PPAT), a novel framework that enhances label-efficient risk estimation by integrating the unbiased LURE estimator with a prediction-powered control variate. By leveraging predictions from powerful black-box models to reduce variance without introducing bias, PPAT not only improves the estimator's performance but also optimizes the selection of test points for labeling. Experimental results demonstrate that PPAT outperforms existing methods in both tabular regression and image classification tasks, achieving more accurate risk estimates with fewer labels and tighter confidence intervals.
PPAT achieves more accurate risk estimates with fewer labels by leveraging predictions from black-box models, transforming the landscape of active testing.
Active testing provides a label--efficient approach to risk estimation by adaptively selecting which test points should be labelled. However, existing estimators fail to exploit the informative predictions of powerful black--box models, even though such predictions are increasingly available in settings where labels remain expensive. To address this, we propose \textbf{Prediction--Powered Active Testing (PPAT)}, a novel label--efficient risk estimation framework that combines the unbiased LURE estimator \citep{farquhar2021statistical} with a prediction--powered control variate. Rather than using proxy predictions as biased pseudo--labels, PPAT uses them to residualise the loss, preserving unbiasedness while reducing variance. Beyond the estimator itself, PPAT also changes which points should be acquired: we derive oracle and practical surrogate--based acquisition rules tailored to reducing the variance of our estimator. Moreover, we establish asymptotic normality for PPAT, yielding asymptotically valid confidence intervals and thus a principled estimate of the uncertainty around our estimates. Across tabular regression and image--classification tasks, PPAT outperforms existing methods in risk estimation, while its confidence intervals attain the target coverage with substantially fewer labels and smaller widths.
