Search papers, labs, and topics across Lattice.
This paper proposes a framework for AI incident monitoring inspired by public health surveillance, addressing the challenge of quantifying AI risk due to incomplete incident reporting and unknown system prevalence. They identify six phases of incident emergence and demonstrate the framework's utility through case studies on autonomous vehicles (leveraging mandatory reporting) and deepfakes. The study shows that domain experts, combined with statistical tools, can determine incident phases, providing a more comprehensive understanding of AI risks.
Current AI incident databases can't quantify real-world risk because they lack data on how often risky systems are deployed and how frequently incidents are reported, leaving us flying blind on AI safety.
Artificial intelligence systems are now deployed at scale across sectors, accompanied by a growing number of real-world incidents ranging from misinformation and cybercrime to autonomous-system failures. Databases of AI incidents index these events, but they cannot measure ``risk''(i.e., a joint measure of likelihood and severity) without additional data regarding the prevalence of risk-associated systems and their incident reporting rates. As a result, policymakers, companies, and the general public lack a means to weigh the benefits of AI against their in-context risks. Inspired by public-health processes, which presume noisy and incomplete disease surveillance, we identify six phases of incident emergence. We demonstrate the framework through a detailed case study of autonomous vehicles, whose mandatory reporting requirements produces reliable incident-rate ground truth expressed in distance traveled. The case study shows that an informed panel of domain experts (e.g., self-driving experts) can combine their domain expertise, incident data, and a collection of statistical and visualization tools to arrive at incident phase determinations serving public needs. We further demonstrate the approach with a deepfake incident case study and chart a path for future research in incident phase determination.
