Search papers, labs, and topics across Lattice.
This paper explores the design of robust risk models to evaluate the societal risks associated with advanced AI systems, addressing a critical gap in AI governance. By synthesizing insights from diverse research traditions and expert discussions, the authors highlight key methodological and institutional challenges that hinder effective risk modeling. The findings culminate in a structured agenda of open questions and priorities for advancing risk assessment practices, emphasizing the need for integration of quantitative modeling with transparent governance mechanisms.
Risk modeling for advanced AI systems is hampered by a lack of rigorous quantitative methods, leaving critical societal assessments in the dark.
We investigate the design of robust risk models to assess societal risks posed by advanced AI systems, an emerging area in AI governance. Many regulatory proposals increasingly require systemic risk assessment, but in the absence of rigorous quantitative methods, the question remains what state of the art risk modeling should look like in practice. We identify the key methodological and institutional challenges that currently limit the adoption of risk modeling. We review five research traditions that inform this problem: probabilistic risk assessment, catastrophic AI risk analysis, cybersecurity risk quantification, Bayesian causal inference, and threshold-based governance. We compare two leading proposals, scenario-based risk estimation and Bayesian network-based threshold setting. Drawing on a workshop with 22 experts and subsequent analysis, we identify a structured agenda of open questions concerning model structure, scope, evidence integration, validation, and governance. We close by outlining priorities for progress, arguing that it will depend on integrating quantitative modeling with independent evaluation, transparent and tiered disclosure, and institutions capable of maintaining and updating risk models over time.
