Search papers, labs, and topics across Lattice.
This paper introduces an efficient training-time model predictive control (MPC) guidance method for humanoid locomotion and manipulation, termed MPC-RL, which integrates a centroidal-dynamics MPC reward formulation to enhance reinforcement learning. The proposed $蟺^n$MPC solver enables parallel processing of time-varying dynamics without the need for extensive problem construction or memory-intensive pre-compilation, significantly reducing training overhead. Experimental results demonstrate that MPC-RL outperforms traditional methods in developing robust locomotion and manipulation skills in humanoid robots.
MPC-RL achieves superior humanoid locomotion and manipulation performance by integrating efficient MPC guidance, challenging the traditional RL training paradigms.
In humanoid motion control, model predictive control (MPC) offers physically grounded prediction and constraint handling, while reinforcement learning (RL) enables robust whole-body skills through large-scale simulation. However, using MPC inside RL often requires time-consuming problem construction or excessive training overhead, making such frameworks difficult to justify in practice. This work studies efficient training-time MPC guidance for humanoid locomotion and manipulation, termed MPC-RL. We introduce a centroidal-dynamics MPC reward formulation that leverages guidance from MPC trajectories in training time. To make this practical in massively parallel RL, we develop $蟺^n$MPC, a parallel-in-horizon and construction-free batched GPU MPC solver that operates directly on time-varying dynamics to avoid high memory usage and pre-compilation. Through a variety of comparative studies and hardware validations, we have found that MPC-RL achieves superior performance in locomotion and manipulation skills. The code base is available at https://github.com/junhengl/mpc-rl.
