Search papers, labs, and topics across Lattice.
This paper introduces a methodology for making understanding explicit and assessable in AI safety decisions, addressing the challenges posed by time-pressure and AI-generated artifacts. By delineating four key objects of understanding鈥攄ecision, decision-frame, safety justification, and system-in-context鈥攖he authors provide a framework for evaluating the adequacy of decision-makers' understanding. The methodology was trialed in two scenarios, revealing that the analyses of understanding significantly influence engineering outcomes, emphasizing the importance of internal and external coherence in safety justifications.
Understanding in AI safety decisions isn't just a checkbox鈥攊t's a generative force that shapes engineering outcomes.
Decision makers need sufficient understanding to make good decisions about complex AI systems. However, AI deployment decisions are increasingly made under time-pressure, and this combined with the use of AI generated artefact creation, can mean that the existence of safety cases and system cards may no longer demonstrate that sufficient understanding exists. Our provisional methodology for making understanding explicit and assessable requires the production of an explicit description of 4 objects of understanding (decision, decision-frame, safety justification, system-in-context) and a justification for the adequacy of this understanding. In addition, the methodology provides a mechanism for describing and evaluating the adequacy of the decision-maker representation of this understanding. It builds on recent developments in safety cases using the Assurance 2.0 framework to operationalise the philosophical basis of understanding from Elgin and Arendt. To assess the methodology we trialled two different scenarios. One scenario, which we investigated through role-based analysis, concerned the risk of scheming in the deployment of an AI coding agent in a robotics company and the other scenario was for the higher uncertainty, more decision-critical argument of'If Anyone Builds It, Everyone Dies'(Yudkowsky and Soares). The trial's central finding, for these two scenarios, is that the methodology could be applied and was found to be generative: we found the analyses that justify sufficiency of understanding (internal coherence, tethering, felicitous falsehoods, external coherence) drives the engineering.
