Search papers, labs, and topics across Lattice.
This paper introduces a framework that integrates deep reinforcement learning (DRL) with model predictive control (MPC) to ensure safety constraints are upheld during the learning process in complex cyber-physical systems. By leveraging offline MPC computations to define a feasible state-action space, the proposed method guarantees that the RL agent's actions remain within safe operational limits, thus preventing potential damage during exploration. Experimental results on a non-linear 1-DoF laboratory testbed show that this approach enables successful exploration and stable policy convergence in real-world scenarios.
Combining DRL with MPC not only enhances safety in exploration but also ensures stable policy convergence in complex physical systems.
Reinforcement learning (RL) enables the synthesis of control policies directly from data, making it highly appealing for complex cyber-physical systems (CPSs) and robotics. A persistent challenge, however, is ensuring strict, hard safety constraints during the active learning phase. In real-world physical systems, violating mechanical limits can cause irreversible damage, necessitating that exploration remains strictly within safe operational regions. We propose a generalized framework that combines the adaptive, high-performance nature of deep reinforcement learning (DRL) with the formal safety guarantees of model predictive control (MPC). Using a mathematical model of the system dynamics, offline MPC computations define a feasible state-action space, representing all safe combinations of system states and control inputs that guarantee constraint satisfaction. During training and deployment, the RL agent's instantaneous actions are projected onto this globally verified feasible set via a safety filter. We systematically evaluate our generalized approach on a non-linear 1-DoF laboratory testbed, demonstrating successful exploration and stable policy convergence on physical hardware.
