Search papers, labs, and topics across Lattice.
This paper investigates the application of shielding techniques for autonomous systems that depend on imperfect perception, addressing the safety risks posed by misclassified sensor readings. By modeling the system as an Interval Partially Observable Markov Decision Process (POMDP) and leveraging finite labeled data to construct confidence intervals for perception outcomes, the authors develop a runtime shield that guarantees safety under specified conditions. Experimental results across four case studies demonstrate that their approach significantly enhances safety compared to existing state-of-the-art methods.
Shielding can ensure safety in autonomous systems even when perception is uncertain, potentially transforming how we manage risks in AI decision-making.
Autonomous systems that rely on learned perception can make unsafe decisions when sensor readings are misclassified. We study shielding for this setting: given a proposed action, a shield blocks actions that could violate safety. We consider the common case where system dynamics are known but perception uncertainty must be estimated from finite labeled data. From these data we build confidence intervals for the probabilities of perception outcomes and use them to model the system as a finite Interval Partially Observable Markov Decision Process with discrete states and actions. We then propose an algorithm to compute a conservative set of beliefs over the underlying state that is consistent with the observations seen so far. This enables us to construct a runtime shield that comes with a finite-horizon guarantee: with high probability over the training data, if the true perception uncertainty rates lie within the learned intervals, then every action admitted by the shield satisfies a stated lower bound on safety. Experiments on four case studies show that our shielding approach (and variants derived from it) improves the safety of the system over state-of-the-art baselines.
