Search papers, labs, and topics across Lattice.
This study evaluates the effectiveness of various model-free reinforcement learning (RL) controllers in managing a nonlinear system subjected to cyberattacks, specifically false data injection and denial-of-service attacks. By analyzing four different RL reward structures, the research identifies that the Lyapunov reward significantly enhances resilience while maintaining low tracking error, while exponential rewards provide a favorable balance under moderate training conditions. The findings reveal that while RL-MPCs exhibit strong steady-state resilience, they demand longer training times compared to the quicker, yet less robust, RL-PID controllers, with Proximal Policy Optimization outperforming Deep Deterministic Policy Gradient in terms of key performance indicator (KPI) variance reduction.
Lyapunov rewards not only improve resilience against cyber threats but also maintain low tracking errors, outperforming other reward types in critical scenarios.
This paper compares the performance of model-free controllers on a nonlinear system under cyberattacks, including false data injection and denial-of-service attacks. Four RL reward types are analyzed for accuracy, cost, and resilience. Results show that the Lyapunov reward offers the best resilience with low tracking error. Exponential mode also provides good trade-offs with acceptable resilience under moderate training conditions. Progressive and linear rewards converge faster but are less robust. RL-MPCs show strong steady-state resilience but require longer training times; RL-PID controllers are faster with significantly less training time. Proximal Policy Optimization outperforms Deep Deterministic Policy Gradient with a significant reduction in KPI variance. This study serves to highlight how well-designed RL rewards can improve performance and resilience against cyber threats.
