Search papers, labs, and topics across Lattice.
This paper introduces a contextual bandit formulation addressing dense arm features, non-linear rewards, and non-stationary correlated reward distributions. To tackle this problem, the authors propose conditionally coupled contextual C3 Thompson sampling for Bernoulli bandits, integrating an enhanced Nadaraya-Watson estimator on an embedding space with Thompson sampling. Empirical evaluations demonstrate that C3 achieves a 5.7% reduction in average cumulative regret across four OpenML datasets and a 12.4% click lift on the Microsoft News Dataset (MIND) compared to existing algorithms.
Thompson Sampling gets a major upgrade with C3, outperforming existing methods by 12.4% in click-through rate on the Microsoft News Dataset by better handling non-stationary correlated rewards.
Contextual bandits are incredibly useful in many practical problems. We go one step further by devising a more realistic problem that combines: (1) contextual bandits with dense arm features, (2) non-linear reward functions, and (3) a generalization of correlated bandits where reward distributions change over time but the degree of correlation maintains. This formulation lends itself to a wider set of applications such as recommendation tasks. To solve this problem, we introduce conditionally coupled contextual C3 Thompson sampling for Bernoulli bandits. It combines an improved Nadaraya-Watson estimator on an embedding space with Thompson sampling that allows online learning without retraining. Empirical results show that C3 outperforms the next best algorithm by 5.7% lower average cumulative regret on four OpenML tabular datasets as well as demonstrating a 12.4% click lift on Microsoft News Dataset (MIND) compared to other algorithms.
