Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the limitations of existing regularization techniques in reinforcement learning by proposing a redesigned approach that enhances policy function smoothing. The authors identify key shortcomings in the original implementation, which struggled with balancing smoothness and expressiveness, and present remedies that lead to improved performance across various tasks. Notably, their modified regularization not only achieves smoother motion but also enhances control performance, demonstrating robustness in sim-to-real applications for quadruped robots under dynamic conditions.
Redesigning regularization can significantly enhance policy smoothing, leading to improved control and robustness in reinforcement learning applications.
This paper proposes a novel regularization design to effectively smooth policy functions in reinforcement learning. While regularization that enhances ``global''Lipschitz continuity was initially considered, it has been limited to ``local''Lipschitz continuity due to a tradeoff between smoothness and expressiveness. However, it has become apparent that the original implementation is cumbersome and does not provide sufficient smoothing, leading to a preference for simpler implementations. This stems from a discrepancy between theory and implementation, and a more appropriate implementation can expect to facilitate smoothing. Therefore, this paper identifies three reasons why the original implementation does not function adequately and provide remedies for them. This modified regularization performs well across multiple tasks and algorithms, successfully achieving smooth motion while improving control performance. Furthermore, by applying it to sim-to-real reinforcement learning for a quadruped robot, it is demonstrated that smooth motion provides robustness against sudden changes in target velocity commands.