Search papers, labs, and topics across Lattice.
This paper introduces DOMD-GLB, a novel algorithm for nonstationary generalized linear bandits that uses discounted online mirror descent for parameter estimation. By leveraging DOMD, the algorithm achieves constant per-round computation and memory costs, a significant improvement over existing MLE-based approaches that require revisiting past observations. The paper provides dynamic regret bounds for DOMD-GLB in both drifting and piecewise-stationary environments, demonstrating its efficiency and effectiveness in handling nonstationarity.
Achieve statistically efficient regret guarantees in nonstationary bandit problems with *constant* per-round computation and memory costs – a feat previously unattainable.
We study nonstationary generalized linear bandits (GLBs), where the expected reward is modeled through a nonlinear link function with an unknown time-varying parameter. This framework encompasses a broad class of reward models, including linear, Bernoulli, and binomial rewards. Existing approaches are predominantly based on maximum-likelihood estimation (MLE), using sliding-window, restart, or discounting mechanisms to handle nonstationarity. Although these methods achieve statistically efficient regret guarantees, they generally require revisiting past observations at every round, which leads to computation and memory costs that grow with time; moreover, several of them rely on a non-convex projection step. In this paper, we propose DOMD-GLB, a new algorithm for nonstationary GLBs that utilizes discounted online mirror descent (DOMD) for parameter estimation, thereby incurring only $O(1)$ computation and memory costs per round. We prove dynamic regret bounds of order $\tilde{O} \big(c_μ^{-1/2} d^{3/4} P_T^{1/4} T^{3/4}\big)$ in drifting environments and $\tilde{O}\big(c_μ^{-1/3} d^{2/3} Γ_T^{1/3} T^{2/3}\big) $in piecewise-stationary environments, where $d$ denotes the feature dimension, $T$ the time horizon, $P_T$ the path length, $Γ_T$ the number of change points, and $c_μ$ a curvature parameter associated with the link function, while substantially improving computational efficiency over prior work. To the best of our knowledge, this is the first algorithm for nonstationary GLBs with per-round computation and memory costs independent of time.
