Search papers, labs, and topics across Lattice.
This paper addresses the high variance issue in off-policy evaluation (OPE) for contextual bandits by proposing Variance Optimal-CEM (VOCEM), an estimator that interpolates between the stable but locally dependent OffCEM and the unbiased but high-variance Doubly Robust (DR) estimator. VOCEM achieves variance optimality by selecting an interpolation coefficient that minimizes variance, yielding an unbiased estimator with variance no greater than either endpoint. Empirical results demonstrate VOCEM's superior stability and robustness across synthetic and large-action benchmarks.
A single interpolation coefficient unlocks variance-optimal off-policy evaluation by seamlessly blending the stability of cluster-level weights with the unbiasedness of action-level importance sampling.
Off-policy evaluation (OPE) for contextual bandit policies becomes challenging when action-level importance weighting incurs excessive variance. Doubly robust (DR) estimation remains unbiased under common support but retains these high-variance action-level weights. A prior estimator, Off-policy evaluation with Conjunct Effect Model (OffCEM), replaces them with more stable cluster-level weights, at the cost of relying on local correctness of the reward model. In this paper, we show that, under the assumptions required by DR and OffCEM, there exists an unbiased family of estimators that interpolates between OffCEM and DR. Building on this result, we propose the Variance Optimal-CEM (VOCEM) estimator, which selects the interpolation coefficient to minimize variance. We derive the population-optimal coefficient in closed form and show that the resulting estimator has variance no larger than either endpoint, OffCEM or DR. Experiments in controlled synthetic settings and on two large-action benchmarks show that VOCEM improves upon both endpoints in all 23 evaluated conditions, exhibiting greater stability and empirical robustness.
