Search papers, labs, and topics across Lattice.
This paper introduces the Feasible Action for Optimal Control (FAOC) framework, which integrates Reinforcement Learning (RL) and Optimal Control (OC) to manage constrained dynamical systems while ensuring safety and feasibility. By employing a novel optimization-based mapping algorithm, FAOC transforms the RL agent's actions into a state-dependent feasible parameter set for the Optimal Control Problem, thus maintaining strict adherence to system constraints without the need for expert-designed action spaces. Experimental results in robot table tennis demonstrate that FAOC significantly enhances both sample efficiency and closed-loop performance compared to existing methods.
FAOC transforms RL actions into guaranteed feasible controls, achieving superior performance in constrained environments without expert intervention.
Operating constrained dynamical systems requires controllers to efficiently solve complex tasks while enforcing recursive feasibility and safety constraints. To address these competing requirements, we present Feasible Action for Optimal Control (FAOC), a novel control framework integrating Reinforcement Learning (RL) and Optimal Control (OC). The key contribution is a computationally efficient, optimization-based mapping algorithm that transforms the RL agent's action from a static abstract set into a state-dependent feasible parameter set of the Optimal Control Problem (OCP), guaranteeing strict satisfaction of the dynamical system's constraints. Thus, FAOC effectively combines the predictable safety of OC with the flexibility of RL. In contrast to prior work, the abstract action space of the RL agent does not require expert or heuristic design, and the OCP formulation is not compromised by the inability of RL to guarantee feasibility. We apply our approach to real-time motion planning for robot table tennis, which encapsulates these challenges. Via simulated experiments, we show that FAOC outperforms state-of-the-art baselines in both sample efficiency and closed-loop performance.
