Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study addresses the challenges of deploying reinforcement learning (RL) control policies on low-cost quadrupedal robots, which suffer from significant transport delays and noisy motor feedback that exacerbate the sim-to-real gap. By employing a biologically inspired approach that integrates a forward model of actuator delays with a time-aware neural network, the researchers achieved robust locomotion even under substantial latency perturbations. The resulting system not only demonstrates effective control but also learns a central pattern generator (CPG) that mimics biological rhythmic gaits, suggesting a novel strategy for enhancing locomotion in cost-constrained robotic platforms.
A time-aware neural network can learn robust locomotion strategies that withstand over 320 ms of actuator latency, bridging the sim-to-real gap for low-cost quadrupedal robots.
Deploying learned control policies on low-cost robotic platforms introduces transport latencies and noisy motor feedback that systematically widens the sim-to-real gap. The chasm of simulation to deployment in hardware lies in the delay of the actuator reaching the commanded position. On platforms such as the Mini Pupper 2, a measured>50 ms transport delay transforms the locomotion task from a standard Markov decision process into a partially observable one. In this paper, we take a biologically inspired approach of handling noisy and delayed feedback to close the sim-to-real gap, thereby expanding the capability of reinforcement learning on cost-constrained hardware. Using a low-cost quadrupedal hardware platform, we find that using a forward model of the average actuator delay, paired with a time-aware neural network results in robust locomotion. Additionally, our time-aware neural network learned a central pattern generator (CPG): a self-sustaining rhythmic gait that is robust to +320 ms latency perturbations, mirroring the CPGs found in the spinal cords of vertebrates. We posit that temporal self-organization may be a general strategy for cost-constrained locomotion.