Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
We address the best arm identification problem in stochastic bandits under adversarial "Shifting Means" perturbations, where only reward gaps are stable, not absolute means. Existing GLRT-based algorithms like Track-and-Stop fail in this setting, necessitating a new approach. Our proposed Importance Weights for Shifting Means ($\mathsf{ISM}$) achieves a sample complexity of $O(K (σ^2 + U^2) Δ_{\min}^{-2} \ln \frac{1}δ)$, matching a derived lower bound.
Adversarial shifts in bandit arm means, not just gaps, break standard best-arm identification algorithms, requiring a novel importance-weighting approach for robust performance.
We study the best arm identification problem in a stochastic environment with a novel form of adversarial perturbations, which we coin Shifting Means. While classically the mean rewards of the $K$ arms are stable in time, in Shifting Means only the gaps $\boldsymbolΔ$ between mean rewards are stable, while their common shift may be determined adversarially in each round. The objective of the learner is to identify the best arm with high probability while minimizing sample complexity (the fixed confidence setting). Handling shifts requires new tools: we show that algorithms employing a Generalized Likelihood Ratio Test (GLRT) stopping rule, including the popular Track-and-Stop, fail under time-varying shifts. Instead, we propose Importance Weights for Shifting Means ($\mathsf{ISM}$). Assuming means bounded by $U$ and $σ^2$-sub-Gaussian rewards, we show $\mathsf{ISM}$ to be $δ$-correct and to enjoy a sample complexity bound of order $K (σ^2 + U^2) Δ_{\min}^{-2} \ln \frac{1}δ$. We also present a matching (up to constant factors) worst-case lower bound and evaluate our results empirically.