Search papers, labs, and topics across Lattice.
This paper investigates the efficacy of multivariate two-sample drift detectors for large-scale e-commerce data, addressing the under-characterization of their empirical behavior on high-cardinality, massive datasets. Through evaluations on public, reproduced, and novel synthetic benchmarks, the study demonstrates that distributed Maximum Mean Discrepancy with Random Fourier Features on Apache Spark offers robust scalability and high detection accuracy (r=0.940, 80.4% TPR, 3.2% FPR) for strong drift regimes. Conversely, per-dimension Kolmogorov-Smirnov tests fail due to statistic saturation, highlighting critical sampling design constraints for industrial operations.
Distributed MMD with Random Fourier Features achieves near-perfect correlation with expected drift magnitude on massive e-commerce data, while per-dimension KS tests falter.
Concept drift threatens production machine learning, yet the empirical behavior of multivariate two-sample drift detectors at scale remains under-characterized. Existing benchmarks rarely address the hundreds of millions of rows and high-cardinality features typical of industrial-operational datasets. We evaluate five multi-column two-sample tests (marginal, projection-based, and kernel embedding methods) across three complementary environments: the Harvard Dataverse, a validated Failing Loudly reproduction (mean absolute error between 0.030 and 0.053), and a novel synthetic-injection benchmark on the 137.5-million-row Trendyol collection-ranking feature table. Testing four drift types across two severity-scope regimes, we demonstrate that distributed Maximum Mean Discrepancy with Random Fourier Features on Apache Spark scales robustly. Averaged over the four drift types in the strong regime and under a calibrated threshold, it achieves a Pearson correlation of r = 0.940 with expected drift magnitude, an 80.4% true positive rate, and a 3.2% false positive rate. Conversely, the per-dimension Kolmogorov-Smirnov test failed due to statistic saturation from ID-like columns under asymmetric sampling, establishing a critical constraint for large-scale sampling design. At weak configurations (realized-flip fractions of at most 0.57%), detectors struggled to reliably discriminate, highlighting the need for future intensity-grid power analyses to distinguish fundamental sensitivity bounds from scalable threshold shifts.
