Search papers, labs, and topics across Lattice.
This paper introduces Open-DiffLoco, an open-source framework that leverages differentiable simulation to efficiently train deployable blind quadruped locomotion policies. By implementing the Short-Horizon Actor-Critic (SHAC) algorithm in MuJoCo XLA, the framework simplifies the training process by using a minimal reward function and removing privileged actor observations, enabling the robot to learn effective walking patterns autonomously. The resulting policy demonstrates robust performance on physical hardware, achieving accurate velocity tracking and adaptability to various terrains with significantly reduced training time and resource requirements.
Open-DiffLoco achieves robust quadruped locomotion with minimal training complexity, outperforming traditional reinforcement learning methods.
Developing deployable locomotion policies through conventional reinforcement learning often requires complex reward engineering and expensive training times. While differentiable simulation offers a highly efficient alternative, open-source tools capable of end-to-end transfer of these policies to physical hardware remain limited. This paper introduces Open-DiffLoco, an open-source framework for training deployable blind quadruped locomotion policies with differentiable simulation. The framework implements the Short-Horizon Actor-Critic (SHAC) algorithm in MuJoCo XLA (MJX) and trains a proprioceptive policy that transfers to real-world hardware. The deployed policy removes privileged actor observations, including base linear velocity, and does not rely on reference trajectories. It also uses a substantially simplified reward function, enabling the robot to discover walking patterns without the complex auxiliary rewards typically used in conventional reinforcement learning pipelines. When deployed on physical hardware (a Unitree Go2 quadruped), the trained policy tracks omnidirectional velocity commands with root-mean-square error below 0.2 m/s, reaches speeds above 1 m/s, and remains robust to uneven terrain and external physical disturbances, such as lateral pushes. Across the reported configurations, training uses under 6 GB of VRAM on a single NVIDIA GeForce RTX 5080 GPU and completes in approximately 20-60 minutes. As an algorithmic extension to SHAC, we propose Jacobian-Augmented Value Estimation (JAVE), which supervises the critic Jacobians to improve early first-order policy-gradient training. To our knowledge, Open-DiffLoco is the first open-source framework for training deployable locomotion policies using differentiable simulation. Deployment videos and source code are available at: https://diffloco.martin-opat.com/
