Search papers, labs, and topics across Lattice.
This paper addresses the problem of Markovian interference in A/B testing of scheduling algorithms in data centers. They introduce mixed Differences-in-Q estimators, leveraging Little's Law, to mitigate bias and variance in A/B tests. Through extensive simulations with non-stationary arrival rates, heterogeneous service rates, and communication delays, the authors demonstrate the robustness and efficacy of their proposed A/B testing approach.
A/B testing scheduling algorithms in data centers is now less biased and more reliable thanks to a new estimator grounded in Little's Law.
In data centers, tasks are dispatched to various servers to evenly distribute the workload. When a data center considers implementing a new scheduling algorithm, it typically conducts an A/B test prior to deployment to assess the real-world impact of this new method. However, a straightforward A/B test might be interfered with so-called ``Markovian''interference. We utilized the Differences-in-Q estimator, as developed by Farias et al. (2022), and introduced mixed Differences-in-Q estimators grounded in Little's Law. We show that our A/B testing methods significantly reduce bias and variance when testing various scheduling policies. Extensive simulations were conducted under scenarios like non-stationary arrival rates, heterogeneous service rates, and communication delays. These simulations highlight the robustness and efficacy of our A/B testing approach.
BAIR