Search papers, labs, and topics across Lattice.
This paper introduces SKooP, a novel reinforcement learning approach that integrates morphological symmetries with a Koopman model learned via autoencoder to enhance policy learning for legged robots. By utilizing Koopman predictions as privileged observations for the critic and incorporating group symmetries into the neural network architecture, SKooP achieves significant improvements in sample efficiency and performance. The results demonstrate that SKooP not only accelerates convergence time but also enhances reward learning across various bipedal locomotion tasks on quadruped robots, showcasing its generalizability to different simulation environments.
SKooP reduces convergence time and boosts performance in legged robot locomotion by leveraging symmetries and advanced Koopman predictions.
Reinforcement learning (RL) algorithms classically suffer from poor sample efficiency. In robotics, a recent line of work has emerged addressing this problem by encoding physics priors in the learning process. However, most of these approaches are validated on well-defined, low-dimensional benchmark systems rather than high-dimensional robots with complex nonlinear dynamics. In this paper, we introduce \textit{SKooP (Symmetric Koopman Predictions)}, an approach combining the advantages of morphological symmetries with those of a Koopman model learned via autoencoder to enhance policy learning. SKooP learns a Koopman model of the system dynamics alongside the policy. The resulting Koopman predictions are used as privileged observations for the critic, allowing the agent to learn based on smoother, more informative features. We also incorporate group symmetries into the actor, critic, encoder and decoder networks to produce a highly equivariant policy. The SKooP approach is validated via in-depth analysis of the learned Koopman models and symmetric policies to showcase how each of these influences the agent's performance. We also show that the learned policies are transferable to different simulation environments. Our results show that SKooP consistently reduces convergence time and increases the learned reward for multiple challenging bipedal locomotion tasks on a quadruped robot. Project page: https://evelyd.github.io/SymmetricKoopmanPredictions/
