Search papers, labs, and topics across Lattice.
The paper introduces Fuz-RL, a fuzzy measure-guided framework for safe reinforcement learning designed to handle multiple sources of uncertainty. Fuz-RL employs a novel fuzzy Bellman operator with Choquet integrals to estimate robust value functions, proving its equivalence to solving distributionally robust safe RL problems without requiring min-max optimization. Experiments on safe-control-gym and safety-gymnasium benchmarks demonstrate that Fuz-RL enhances the safety and control performance of existing safe RL baselines under observation, action, and dynamics uncertainties.
Ditch the min-max: Fuz-RL offers a fuzzy-measure guided approach to safe RL that achieves distributional robustness without complex optimization.
Safe Reinforcement Learning (RL) is crucial for achieving high performance while ensuring safety in real-world applications. However, the complex interplay of multiple uncertainty sources in real environments poses significant challenges for interpretable risk assessment and robust decision-making. To address these challenges, we propose Fuz-RL, a fuzzy measure-guided robust framework for safe RL. Specifically, our framework develops a novel fuzzy Bellman operator for estimating robust value functions using Choquet integrals. Theoretically, we prove that solving the Fuz-RL problem (in Constrained Markov Decision Process (CMDP) form) is equivalent to solving distributionally robust safe RL problems (in robust CMDP form), effectively avoiding min-max optimization. Empirical analyses on safe-control-gym and safety-gymnasium scenarios demonstrate that Fuz-RL effectively integrates with existing safe RL baselines in a model-free manner, significantly improving both safety and control performance under various types of uncertainties in observation, action, and dynamics.
