Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper extends the $\KLinf$-UCB algorithm to a broad nonparametric class of reward distributions in stochastic multi-armed bandits, proving its asymptotic optimality in expectation. The authors derive a novel upper bound on the regret tail probability of this algorithm, providing regret-tail guarantees for both bounded-support and heavy-tailed bandit models. For finitely-supported reward distributions, the upper bound matches the known lower bound, offering a unified and tight characterization of regret tails for KL-based UCB algorithms.
Asymptotically optimal bandit algorithms can still exhibit heavy regret tails, but this work provides a unified and tight characterization of these tails for KL-based UCB algorithms even beyond parametric models.
We study the tail behavior of regret in stochastic multi-armed bandits for algorithms that are asymptotically optimal in expectation. While minimizing expected regret is the classical objective, recent work shows that even such algorithms can exhibit heavy regret tails, incurring large regret with non-negligible probability. Existing sharp characterizations of regret tails are largely restricted to parametric settings, such as single-parameter exponential families. In this work, we extend the $\KLinf$-UCB algorithm of to a broad nonparametric class of reward distributions satisfying mild assumptions, and establish its asymptotic optimality in expectation. We then analyze the tail behavior of its regret and derive a novel upper bound on the regret tail probability. As special cases, our results recover regret-tail guarantees for both bounded-support and heavy-tailed (moment-bounded) bandit models. Moreover, for the special case of finitely-supported reward distributions, our upper bound matches the known lower bound exactly. Our results thus provide a unified and tight characterization of regret tails for asymptotically optimal KL-based UCB algorithms, going beyond parametric models.