Search papers, labs, and topics across Lattice.
This paper introduces a method for integrating locomotion and manipulation in robotic systems by utilizing Sample-based Model Predictive Control (SMPC) to generate extensive offline datasets in simulation. By training an off-policy reinforcement learning agent on these datasets with sparse task rewards, the authors significantly reduce the learning time and eliminate the need for manual reward shaping. The approach not only enhances the performance of the learned policies beyond the original optimal control teacher but also demonstrates successful deployment across various robotic platforms, including quadrupeds and humanoids.
Sparse rewards can now drive complex loco-manipulation skills, enabling robots to learn faster and perform better than traditional methods.
Integrating locomotion and manipulation is essential for robot autonomy, but scaling standard Reinforcement Learning (RL) to complex tasks is severely bottlenecked by the slow, manual process of dense reward shaping. To bypass this limitation, we leverage Sample-based Model Predictive Control (SMPC) entirely in simulation as an automated, rapidly tunable expert to generate massive offline datasets. Because this data solves the fundamental exploration problem, we can train an off-policy RL agent using purely sparse task rewards, drastically reducing the time required to learn new skills and eliminating the need for manual tuning. Integrating this high-level agent with a low-level dynamic stability controller yields more optimal behaviors that strictly align with true task objectives, ultimately allowing the learned policies to surpass the original optimal control teacher. We validate the robustness of this sim-to-real framework by successfully deploying complex loco-manipulation skills across different morphologies, including an arm-equipped Spot quadruped and a G1 humanoid.
