Qualifying and Quantifying Risk under the EU AI Act

The EU AI Act uses a risk-based approach to regulate AI systems, calibrating the intensity of regulation according to the risks they pose. While the AI Act's definition of 'risk' (as the combination of the probability and severity of harm) implies quantification, the AI Act focuses on risks to fundamental rights, thereby engaging a typically qualitative perspective. In this piece, we address this tension using a two-step framework under which the EU AI Act balances the protection of fundamental rights, the legitimate purposes of providers and deployers, and the impacts of regulatory measures on providers, deployers, and regulators. We discuss this framework against the backdrop of potential approaches to quantifying risks, with a specific focus on defining and measuring the main components of the concept of risk: 'probability', 'severity', and their 'combination'. We suggest that the protection of fundamental rights and risk quantification can be aligned within the proposed framework. In particular, the AI Act implies a balancing analysis that uses what we call a 'severity-first' approach in which the severity of a harm is considered before the harm's probability of occurring. The integrated framework not only helps to clarify the AI Act's risk-based approach, but can also inform technical and implementation choices. Finally, we discuss the potential for 'risk hacking' if risk quantification is left to AI providers and deployers, who may manipulate the approach, which could lead to the underclassification of AI systems and subsequent regulatory shortcuts.