Search papers, labs, and topics across Lattice.
This paper introduces a framework for classifying AI incident trajectories that disentangles reporting biases, deployment growth, and harm frequency. The framework uses a structured monitoring question format (SORT), tiered estimation procedures, and LLM-assisted incident matching to analyze public AI incident data. Applying this framework reveals the necessity of exposure estimation for accurate risk assessment as AI deployments increase.
Public AI incident databases are misleading: this framework disentangles reporting biases from actual harm trends, enabling more informed AI governance.
Public AI incident database counts conflate changes in reporting propensity, deployment growth, and shifts in harm frequency per unit of exposure. These issues introduce significant uncertainties challenging public and corporate policy frameworks centred on realized risks. We propose a simple framework that establishes clear points of inquiry, separately estimates exposure from harm-rate trends, and then classifies into meaningful trajectory categories for governance decisions. The framework combines a structured monitoring question format (SORT) to clarify coverage decisions, a tiered estimation procedure calibrated to available evidence, and LLM-assisted incident matching against public databases. Applied to various monitoring questions, we draw conclusions regarding the monitoring ecosystem more broadly: Providing an essential interpretative classification, determining what can and cannot be claimed, and establishing that exposure estimation is required as AI deployments become increasingly common.
