Search papers, labs, and topics across Lattice.
This paper critiques existing risk assessment methodologies for AI systems, highlighting their inadequacies in addressing the unique safety, socio-economic, and ethical challenges posed by AI deployment. It introduces a novel end-to-end AI risk framework that incorporates Operational Design Domains (ODD) to delineate the specific conditions under which AI systems are expected to operate safely. The key finding suggests that a well-defined operational envelope can enhance risk assessment and safety mitigation strategies, ultimately leading to more reliable AI deployments.
A well-defined operational envelope could revolutionize how we assess and mitigate risks in AI systems, ensuring safer deployments.
Novel safety, socio-economic, and ethical harms arising from the deployment of AI-based systems have led to a breadth of work seeking to map, measure, and mitigate against newly found risks. These works have heavily leveraged techniques and terminology from the fields of System Safety Engineering and Cybersecurity, yet they have fallen short in accounting for the limitations and nuances that reduce the efficacy and correct application of adopted methodologies. Furthermore, misuse of terminology entailing compliance with established safety and security properties can mislead stakeholders with regard to the claims an AI system satisfies and provide a false sense of safety. In this paper, we seek to align overlapping, AI-adjacent communities on a consistent and comprehensive assurance terminology crucial for the safe deployment of AI-based systems. We outline why previous attempts to adapt risk assessment techniques and terminology from the safety and security fields have been insufficient. We then propose a novel end-to-end AI risk framework that integrates the concept of an Operational Design Domains (ODD), initially introduced for ADS (Automated Driving Systems) [1], for more general AI-based systems. The purpose of an ODD is to provide a description of the specific operating conditions for which an AI-system is designed to properly behave, thus outlining the safety envelope for which system hazards and harms can be determined against. We believe that by defining a more concrete operational envelope, developers and auditors can better assess potential risks and required safety mitigations for AI-based systems.
