Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper analyzes the convergence properties of success conditioning, a prevalent policy optimization strategy in reinforcement learning. The authors prove that success conditioning converges to an optimal policy for a wide range of Markov Decision Processes. Specifically, they derive convergence rates of $\mathcal{O}(1/\varepsilon^p)$ for discounted MDPs and $\mathcal{O}(\log(1/\varepsilon))$ for single-period MDPs, offering theoretical guarantees for this widely used technique.
Success conditioning, a staple in RL, is formally proven to converge to optimal policies across broad MDP classes, with explicit convergence rates established.
Success conditioning is a strategy for improving decision-making policies in stochastic environments; it updates a policy by increasing the probability of taking actions that yield successful outcomes. Success conditioning is common to many reinforcement learning applications, yet its limiting behavior and convergence rates are not well understood. In this work, we demonstrate that success conditioning converges to an optimal policy on a broad class of Markov decision processes (MDPs). We also derive convergence rates in some common settings. For discounted MDPs, we prove convergence within $\mathcal{O}(1/\varepsilon^p)$ iterations to an $\varepsilon$-optimal policy, where the exponent $p$ depends on problem data. For single-period MDPs, such a policy is obtained within $\mathcal{O}(\log(1/\varepsilon))$ iterations.