Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses policy learning challenges in digital experimentation, characterized by weak signals, rich covariates, and large data. The authors formalize this by modeling treatment-effect estimates as Gaussian observations with bounded signal-to-noise ratios across covariate partitions, proving that optimal policies are generally not learnable and optimal policy value learning is impractically slow. However, by assuming smooth treatment effects, they derive minimax-adaptive policies using linear smoothers that achieve vanishing welfare regret, validated by Netflix experiments showing personalized linear-smoothing policies outperform unpersonalized ones.
Smoothness in treatment effects unlocks near-optimal policy learning in high-dimensional, low-signal digital experimentation.
Policy learning in digital experimentation faces three challenges: weak signal-to-noise ratios, rich covariate spaces, and massive data volumes. We formalize this regime by modeling treatment-effect estimates from increasingly fine covariate partitions as Gaussian observations with bounded signal-to-noise ratios. We establish that, in general, the optimal treatment policy is not learnable in this setting. Even learning the optimal policy value suffers from impractically slow rates. However, when treatment effects vary smoothly, we derive minimax-adaptive policies based on linear smoothers that achieve vanishing welfare regret. We demonstrate the practical value of our framework by applying it to large-scale real-world experiments at Netflix, showing that personalized linear-smoothing policies can dominate unpersonalized policies even in this challenging empirical setting.