Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Safe-RULE, a novel framework for safe reinforcement unlearning that addresses vulnerabilities in offline safe reinforcement learning (Safe RL) caused by data poisoning attacks. By effectively removing the influence of malicious samples without the need for complete retraining, Safe-RULE maintains both task performance and safety constraints. Experimental results across benchmark Safe RL tasks show a significant improvement in safety performance against such attacks, highlighting its potential for enhancing safety in critical applications like robotics.
Safe-RULE effectively neutralizes data poisoning threats in offline Safe RL, enhancing safety without the need for retraining from scratch.
Offline safe reinforcement learning (Safe RL) enables policy learning without online interactions, making it suitable for safety-critical systems such as robotics systems. However, its reliance on static datasets exposes offline Safe RL to data poisoning attacks, where adversaries inject malicious samples that compromise safety and induce unsafe policy behavior. In this work, we propose a new learning paradigm, named safe reinforcement unlearning (Safe-RULE), used as a defense framework to remove the influence of poisoned data without retraining from scratch or requiring access to the original training environment. We further extend reinforcement unlearning to offline Safe RL by explicitly accounting for both task performance and safety constraints during the unlearning process. Experiments across benchmark Safe RL tasks demonstrate that our approach effectively enhances safety performance against data poisoning attacks.