Search papers, labs, and topics across Lattice.
This paper analyzes Thompson Sampling (TS) in misspecified bandit environments, focusing on the dynamics of posterior beliefs and regret. It provides a complete dynamic classification of posterior evolution in a misspecified two-armed Gaussian bandit, identifying regimes of correct/incorrect concentration and persistent belief mixing. The authors then generalize this analysis using a stochastic stability framework, representing posterior evolution as a Markov process on the belief simplex to classify ergodic and transient behaviors and provide inductive dimensional reductions of the posterior dynamics.
Thompson Sampling under model misspecification exhibits surprisingly rich dynamics, including persistent belief mixing, revealing a need to rethink its robustness in real-world applications.
Dynamic decision-making under model uncertainty is central to many economic environments, yet existing bandit and reinforcement learning algorithms rely on the assumption of correct model specification. This paper studies the behavior and performance of one of the most commonly used Bayesian reinforcement learning algorithms, Thompson Sampling (TS), when the model class is misspecified. We first provide a complete dynamic classification of posterior evolution in a misspecified two-armed Gaussian bandit, identifying distinct regimes: correct model concentration, incorrect model concentration, and persistent belief mixing, characterized by the direction of statistical evidence and the model-action mapping. These regimes yield sharp predictions for limiting beliefs, action frequencies, and asymptotic regret. We then extend the analysis to a general finite model class and develop a unified stochastic stability framework that represents posterior evolution as a Markov process on the belief simplex. This approach characterizes two sufficient conditions to classify the ergodic and transient behaviors and provides inductive dimensional reductions of the posterior dynamics. Our results offer the first qualitative and geometric classification of TS under misspecification, bridging Bayesian learning with evolutionary dynamics, and also build the foundations of robust decision-making in structured bandits.
