Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper derives closed-form power formulas for prediction-powered inference (PPI) by characterizing the asymptotic variance of the PPI estimator and applying Wald test inversion. The work addresses the practical question of determining the required labeled sample size when using AI/ML models with high predictive power in downstream statistical inference. The key finding is that the reduction in required labeled samples scales roughly with the R2 between predictions and ground truth, which is validated through simulations and biomedical applications.
Unlock massive savings in labeled data: prediction-powered inference slashes sample size requirements by roughly the R2 of your AI/ML model.
Modern studies increasingly leverage outcomes predicted by machine learning and artificial intelligence (AI/ML) models, and recent work, such as prediction-powered inference (PPI), has developed valid downstream statistical inference procedures. However, classical power and sample size formulas do not readily account for these predictions. In this work, we tackle a simple yet practical question: given a new AI/ML model with high predictive power, how many labeled samples are needed to achieve a desired level of statistical power? We derive closed-form power formulas by characterizing the asymptotic variance of the PPI estimator and applying Wald test inversion to obtain the required labeled sample size. Our results cover widely used settings including two-sample comparisons and risk measures in 2x2 tables. We find that a useful rule of thumb is that the reduction in required labeled samples relative to classical designs scales roughly with the R2 between the predictions and the ground truth. Our analytical formulas are validated using Monte Carlo simulations, and we illustrate the framework in three contemporary biomedical applications spanning single-cell transcriptomics, clinical blood pressure measurement, and dermoscopy imaging. We provide our software as an R package and online calculators at https://github.com/yiqunchen/pppower.
Johns Hopkins