Search papers, labs, and topics across Lattice.
This paper introduces a semi-supervised sequential hypothesis testing framework that leverages predictions on unlabeled data to improve statistical power, even with inaccurate predictions. They construct an e-statistic-based sequential test that is anytime valid under label or concept shift assumptions. Experiments on synthetic data and LLM evaluation demonstrate power gains compared to baselines, even with limited unlabeled data and low prediction accuracy.
Unlabeled data can significantly boost the power of hypothesis tests, even when predictions are noisy, offering a practical way to improve statistical inference in data-scarce regimes.
We introduce a testing-by-betting framework that leverages predictions on unlabeled data to enhance the power of sequential hypothesis testing. Given limited samples from the joint distribution of $(X,Y)$, and additional unlabeled samples from the marginal of $X$, we ask how unlabeled data can be used to hypothesize about the distribution of $Y$, and the conditional distribution of $Y\mid X$. We introduce an e-statistic and use it to construct a sequential test. Under standard distributional assumptions -- label shift or concept shift -- we establish that the test is anytime valid. Furthermore, we show that for binary data, the e-statistic has non-trivial power. Crucially, our approach retains these properties even when the underlying predictions are inaccurate. Through simulations and applications to large language models evaluation, we demonstrate power gains over baseline approaches, including prediction-powered inference. These gains persist even with relatively limited unlabeled data and when predictions have low accuracy due to weak correlation between $X$ and $Y$.
