Search papers, labs, and topics across Lattice.
This paper investigates KL-regularized contextual bandits and episodic reinforcement learning (RL) in the context of model misspecification, a scenario where traditional guarantees fail. By introducing KL misspecification formulations, the authors derive high-probability KL-regret guarantees that explicitly account for misspecification, thus extending the applicability of existing methods. The findings demonstrate that the standard realizable KL-regularized setting is a special case of their broader framework, providing new insights into the performance of regression-based algorithms under misspecified conditions.
KL-regularization can still yield high-probability guarantees even when models are misspecified, challenging the assumption that realizability is necessary for effective learning.
We study KL-regularized contextual bandits and episodic reinforcement learning (RL) under general function approximation with model misspecification. Existing guarantees rely on realizability and therefore do not extend to misspecified models, where classical regret bounds may fail. This work introduces KL misspecification formulations for contextual bandits and episodic RL and analyzes regression-based algorithms with Gibbs policy updates. High-probability KL-regret guarantees with explicit misspecification terms are established, recovering the standard realizable KL-regularized setting as a special case.
