Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Guided Riemannian Optimization (GuRO), a novel framework that combines Model Predictive Control (MPC) with reinforcement learning (RL) to enhance decision-making in high-dimensional, nonlinear systems. By employing a curvature-aware optimization approach in a Riemannian parameter space, GuRO effectively navigates the challenges of non-convex loss landscapes, leading to improved convergence and performance. Experimental results on quadruped control tasks show that GuRO outperforms established methods like TRPO and SAC, achieving higher returns and faster training times.
Integrating MPC with RL using curvature-aware optimization leads to faster convergence and superior performance in complex robotic tasks.
Decision-making in high-dimensional, nonlinear systems remains a central challenge in robotics. While model-based methods like Model Predictive Control (MPC) offer sample efficiency and interpretability, their performance degrades when the dynamics model is inaccurate or long-horizon predictions are required. Conversely, model-free reinforcement learning (RL) learns policies directly from interaction but suffers from high sample complexity and unstable optimization. Recent advances in sequence modeling have inspired transformer-based decision-making frameworks that can unify MPC and RL, but their training typically faces significant optimization challenges due to highly non-convex loss landscapes. In this work, we propose a novel framework that integrates MPC with RL in a sequence decision-making framework and leverages a curvature-aware optimization to efficiently tackle non-convex loss landscapes. MPC provides predictions of locally optimal trajectories that guide the decision transformer, removing the need for extensive offline pretraining. To address the slow and unstable convergence of traditional optimizers, we train the policy in a Riemannian parameter space using an efficient Riemannian (curvature-aware) method, leading to faster and more robust optimization. We evaluate our framework on high-dimensional quadruped control tasks and demonstrate consistent improvements over strong baselines, including TRPO, SAC, and Online Decision Transformer, achieving higher returns and faster convergence.