Search papers, labs, and topics across Lattice.
This paper introduces minimax deep deterministic policy gradient (MMDDPG), a reinforcement learning framework designed to train policies robust to adversarial disturbances in continuous control tasks. MMDDPG formulates training as a minimax game between a user policy and an adversarial disturbance policy, where the user minimizes the objective and the adversary maximizes disturbances. To stabilize this interaction, a fractional objective balances task performance and disturbance magnitude, preventing overly aggressive adversarial behavior. Experiments on MuJoCo environments show MMDDPG significantly improves robustness against external force perturbations and model parameter variations.
Make your RL agents disturbance-proof: a fractional objective stabilizes minimax training, leading to policies that shrug off external forces and model uncertainties.
Reinforcement learning (RL) has achieved remarkable success in a wide range of control and decision-making tasks. However, RL agents often exhibit unstable or degraded performance when deployed in environments subject to unexpected external disturbances and model uncertainties. Consequently, ensuring reliable performance under such conditions remains a critical challenge. In this paper, we propose minimax deep deterministic policy gradient (MMDDPG), a framework for learning disturbance-resilient policies in continuous control tasks. The training process is formulated as a minimax optimization problem between a user policy and an adversarial disturbance policy. In this problem, the user learns a robust policy that minimizes the objective function, while the adversary generates disturbances that maximize it. To stabilize this interaction, we introduce a fractional objective that balances task performance and disturbance magnitude. This objective prevents excessively aggressive disturbances and promotes robust learning. Experimental evaluations in MuJoCo environments demonstrate that the proposed MMDDPG achieves significantly improved robustness against both external force perturbations and model parameter variations.
