Search papers, labs, and topics across Lattice.
This paper introduces VIGIL, a runtime enforcement framework designed to ensure compliance with behavioral specifications in AI agent skills, addressing the challenge of contextual granularity in monitoring agent actions. By developing a novel policy language that captures context-specific enforcement requirements and employing symbolic evaluation rules, VIGIL can effectively detect violations that arise from complex interactions across multiple actions. The framework demonstrates impressive performance, achieving over 95% recall in detecting policy violations while maintaining a false-positive rate below 10% across various real-world tasks involving LLM agents.
VIGIL achieves over 95% recall in detecting policy violations in AI agent skills, significantly enhancing runtime enforcement capabilities.
Agentic systems increasingly act through third-party skills, allowing model-generated decisions to affect files, communication channels, and cyber-physical devices. These skills often include natural-language specifications that define access permissions, disclosure limits, execution privileges, and required preconditions. Although such specifications describe the intended boundaries of skill behavior, they do not by themselves provide executable runtime enforcement. Enforcing them raises a contextual granularity challenge: even when a policy is written for a particular task context, a monitor must still decide which events to observe, what state to retain, how far across the execution to reason, and where to intervene. Choosing the wrong granularity can either block benign executions or miss violations that emerge only across multiple actions. Most existing enforcement mechanisms, however, assume a fixed event model or enforcement point. In this work, we present VIGIL, an end-to-end runtime enforcement framework for agentic systems. VIGIL checks an agent's actual execution trace against behavioral policies from skill specifications, operator-defined constraints, and global rules spanning multiple skills. To make such policies executable, VIGIL introduces a policy language that captures context-specific enforcement requirements over agent-tool events, including temporal dependencies, argument constraints, and value-flow conditions. The language is paired with symbolic evaluation rules that translate policies into SMT constraints over finite traces, allowing VIGIL to detect violations that depend on event order, argument relationships, or cross-call value flow rather than relying on fixed single-call filters. On real LLM-agent runs spanning office-document, operational, and engineering tasks, VIGIL detects policy violations with over 95% recall and a false-positive rate below 10%.