Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper critically evaluates the state of AI incident governance, identifying significant inconsistencies in definitions, classifications, and monitoring practices across existing frameworks. By analyzing regulatory and independent efforts, the authors reveal that these discrepancies hinder effective incident management and analysis, ultimately compromising safety post-deployment. The key finding emphasizes the urgent need for standardized approaches to improve the depth and accuracy of incident data collection and reporting in AI systems.
Inconsistent definitions and classifications in AI incident governance could undermine the effectiveness of safety measures after deployment.
AI systems may produce failures after deployment that pre-deployment safety assessments do not anticipate. Managing these failures requires what we refer to as adequate \textit{AI incident governance}, where having good definitions, taxonomies, monitoring practices, reporting mechanisms, and incident analysis is essential. We examine existing frameworks related to AI incident governance by regulatory bodies and independent efforts, and find that while there are frameworks that describe how individual functions can be performed, there is a lack of consistency within the aspects of definitions, classification, monitoring, and reporting. These inconsistencies apply to the types of incident data that is collected and reported, the ways in which they are categorised, and as a result, the depth, representativeness, and accuracy of analysis that can be performed.