Search papers, labs, and topics across Lattice.
This paper introduces a formal framework for runtime governance of AI agents, focusing on evaluating execution paths to enforce compliance policies. It models policies as deterministic functions that map agent identity, partial path, proposed action, and organizational state to a violation probability, generalizing prompt-level instructions and static access control. The authors present policy examples inspired by the AI Act and discuss a reference implementation, highlighting open challenges like risk calibration.
Current AI agent governance methods are too static; runtime evaluation of execution paths is necessary for effective, path-dependent policy enforcement.
AI agents -- systems that plan, reason, and act using large language models -- produce non-deterministic, path-dependent behavior that cannot be fully governed at design time, where with governed we mean striking the right balance between as high as possible successful task completion rate and the legal, data-breach, reputational and other costs associated with running agents. We argue that the execution path is the central object for effective runtime governance and formalize compliance policies as deterministic functions mapping agent identity, partial path, proposed next action, and organizational state to a policy violation probability. We show that prompt-level instructions (and "system prompts"), and static access control are special cases of this framework: the former shape the distribution over paths without actually evaluating them; the latter evaluates deterministic policies that ignore the path (i.e., these can only account for a specific subset of all possible paths). In our view, runtime evaluation is the general case, and it is necessary for any path-dependent policy. We develop the formal framework for analyzing AI agent governance, present concrete policy examples (inspired by the AI act), discuss a reference implementation, and identify open problems including risk calibration and the limits of enforced compliance.
