Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SMC-ES, a novel simulation-based methodology that synthesizes control policies with formal guarantees of performance, safety, and robustness for autonomous cyber-physical systems. By integrating Evolutionary Strategies with Statistical Model Checking, the approach ensures that synthesized policies come with a certificate that bounds the probability of violating specified properties. Experimental results on continuous control tasks demonstrate that SMC-ES achieves competitive performance compared to leading model-free and Safe-DRL baselines, albeit with an acceptable increase in computational cost.
Formal guarantees of safety and robustness in control policies can now be synthesized automatically, challenging the limitations of traditional learning-based methods.
The deployment of autonomous cyber-physical systems in safety-critical environments requires closed-loop control strategies (i.e., policies) that are not only performant but also provably safe and robust. While learning-based methodologies such as Reinforcement Learning offer flexible and scalable approaches to automatically synthesize such controllers, they typically lack the formal guarantees necessary for safe deployment. To bridge this gap, we propose a novel simulation-based methodology to automatically synthesize policies with formal guarantees regarding performance, safety, and robustness specifications. Specifically, given a set of properties to verify, a confidence parameter $\delta$ and an allowable failure probability $\varepsilon$, our method guarantees that the synthesized policy comes with a certificate: with confidence at least $1 - \delta$, the probability of encountering a scenario where the given properties are violated is at most $\varepsilon$. We demonstrate the feasibility of our approach by developing SMC-ES, an algorithm that integrates Evolutionary Strategies with Statistical Model Checking-based verification. We evaluate SMC-ES on a suite of continuous control tasks using Gymnasium and Safety Gymnasium testbeds. Results show that, at the price of a sustainable increase in computational cost, our algorithm provides formal guarantees regarding performance, safety, and robustness specifications, while performing competitively against leading model-free Deep Reinforcement Learning (DRL) and Safe-DRL baselines.