Search papers, labs, and topics across Lattice.
This paper introduces a computationally efficient reinforcement learning algorithm for Markov Decision Processes with Multinomial Logit function approximation that achieves instance-wise optimal regret bounds. The key innovation is a variance-adaptive analysis that explicitly accounts for the variability in learner-environment interactions, leading to tighter regret guarantees compared to worst-case analysis. Empirical results confirm that the proposed method learns optimal policies more efficiently than existing approaches.
Instance-optimality in MNL-based RL is now achievable, thanks to a new algorithm that adapts to the variance of learner-environment interactions.
Reinforcement learning with multinomial logistic (MNL) function approximation has become an important framework due to its flexibility and broad applicability. While existing studies have established regret guarantees under worst-case analysis, they do not capture how performance depends on the variability of the interaction between the learner and the environment. In this paper, we develop a new theoretical analysis for MNL-based Markov decision processes that yields explicit variance-adaptive regret bounds. Our algorithm is computationally efficient and achieves the instance-wise optimal rate of regret, narrowing the gap between upper and lower bounds. Our numerical experiments validate that our method learns optimal policies more efficiently than conventional approaches.
