Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper benchmarks four heterogeneous treatment effect (HTE) estimators—S-Learner, T-Learner, X-Learner (with LightGBM), and Causal Forest—on the Criteo Uplift v2.1 dataset (13.98M records) for marketing uplift modeling. The S-Learner achieves the highest Qini score (0.376), capturing 77.7% of incremental conversions within the top 20% of targeted customers. SHAP analysis identifies feature f8 as the primary driver of HTE, and Causal Forest provides uncertainty estimates, identifying confident persuadables and sleeping dogs.
S-Learners beat more complex meta-learners and causal forests in a large-scale marketing uplift modeling benchmark, suggesting simpler methods can be surprisingly effective for heterogeneous treatment effect estimation.
Estimating Conditional Average Treatment Effects (CATE) at the individual level is central to precision marketing, yet systematic benchmarking of uplift modeling methods at industrial scale remains limited. We present UpliftBench, an empirical evaluation of four CATE estimators: S-Learner, T-Learner, X-Learner (all with LightGBM base learners), and Causal Forest (EconML), applied to the Criteo Uplift v2.1 dataset comprising 13.98 million customer records. The near-random treatment assignment (propensity AUC = 0.509) provides strong internal validity for causal estimation. Evaluated via Qini coefficient and cumulative gain curves, the S-Learner achieves the highest Qini score of 0.376, with the top 20% of customers ranked by predicted CATE capturing 77.7% of all incremental conversions, a 3.9x improvement over random targeting. SHAP analysis identifies f8 as the dominant heterogeneous treatment effect (HTE) driver among the 12 anonymized covariates. Causal Forest uncertainty quantification reveals that 1.9% of customers are confident persuadables (lower 95% CI > 0) and 0.1% are confident sleeping dogs (upper 95% CI < 0). Our results provide practitioners with evidence-based guidance on method selection for large-scale uplift modeling pipelines.