Search papers, labs, and topics across Lattice.
This paper introduces ATACOM Directional Constraints (ATACOM-DC), an enhancement of the ATACOM framework that integrates with existing Reinforcement Learning (RL) algorithms to enforce safety constraints derived from prior knowledge or learned from data. By implementing directional constraints that selectively activate enforcement based on the agent's proximity to constraint boundaries, the method significantly improves the safety-performance trade-off in RL tasks. Evaluations across various robotic control tasks demonstrate that ATACOM-DC effectively reduces constraint violations while maintaining high task performance, addressing a critical challenge in safe RL deployment.
Introducing directional constraints can drastically enhance the safety-performance balance in reinforcement learning, allowing for more efficient exploration without compromising task success.
Reinforcement Learning has revolutionized the landscape of robotic research, allowing robust learning of complex robotic skills in simulation. However, real-world deployment in open-ended environments requires strong safety guarantees to prevent dangerous or harmful behaviors. Safe Reinforcement Learning methods address this requirement by enforcing safety constraints. Nevertheless, learning under constraints often reduces learning speed and could lead to suboptimal task performance, as the agent must solve a more complex constrained optimization problem compared to unconstrained settings. To tackle this issue, in this work, we propose an extension of the ATACOM framework, a state-of-the-art reliable safety layer that can be integrated with existing Reinforcement Learning algorithms to enforce constraints derived from prior knowledge of the system or learned directly from data. Our proposed method, named ATACOM Directional Constraints (ATACOM-DC), significantly improves the safety-performance trade-off by introducing directional constraints that distinguish between actions approaching and moving away from constraint boundaries, activating constraint enforcement only when necessary. We evaluate our method across a range of challenging robotic control tasks in simulation, analyzing both constraint-violation costs and achieved task performance. Code and additional material at https://atacom-dc.robot-learning.net.
