Search papers, labs, and topics across Lattice.
This paper introduces a novel framework for quadruped locomotion that utilizes Signal Temporal Logic (STL) to specify distinct gait behaviors through parameterized constraints. By implementing a reward shaping mechanism based on STL, the authors enable reinforcement learning agents to navigate a dense reward landscape that enhances interpretability and control over gait dynamics. The results demonstrate that STL-shaped rewards significantly improve velocity tracking and stability during training compared to traditional hand-crafted reward functions, as validated on Google's Barkour robot in the MuJoCo XLA environment.
STL-shaped rewards lead to tighter velocity tracking and more stable training for quadruped locomotion, outperforming traditional hand-crafted methods.
Reinforcement learning (RL) for quadruped locomotion commonly depends on fixed, hand-crafted, and Markovian reward functions that limit both interpretability of learned policies and lack explicit control over gait behaviors. We introduce a framework where distinct gaits are specified using parameterized constraints expressed in Signal Temporal Logic (STL). These include safety bounds, gait synchronization constraints, command tracking, and actuation bounds. From these specifications, we develop a reward shaping mechanism that provides learning agents a dense, continuous reward landscape that encodes desired behavior. We define parametric STL templates for three speed regimes (walking-trot, trot, bound), calibrate their parameters from reference rollouts, and compute rewards from using smooth approximations of STL robustness over the rollouts. The generated rewards can be used to provide shaped gradients compatible with Proximal Policy Optimization (PPO). We instantiate the approach on Google's Barkour quadruped robot in MuJoCo XLA (MJX). We use parallelization within the simulator to improve training speeds and use domain randomization to robustify learned policies. We show that compared to a baseline of hand-crafted rewards, the STL-shaped rewards yield tighter velocity tracking and more stable training. Videos can be found on our project website: https://stl-locomotion.github.io/.
BAIR