Search papers, labs, and topics across Lattice.
AgentSCOPE introduces a framework, the Privacy Flow Graph, to decompose agentic workflows into information flows annotated with Contextual Integrity parameters, enabling fine-grained privacy violation tracing. The benchmark of 62 multi-tool scenarios across eight regulatory domains reveals that privacy violations occur in over 80% of scenarios, even when final outputs appear clean. The study identifies the tool-response stage as the primary source of privacy violations, highlighting the limitations of output-level privacy evaluations.
Agentic systems leak sensitive data in 80% of workflows, even when the final output seems safe, because current privacy evaluations miss intermediate steps.
Agentic systems are increasingly acting on users'behalf, accessing calendars, email, and personal files to complete everyday tasks. Privacy evaluation for these systems has focused on the input and output boundaries, but each task involves several intermediate information flows, from agent queries to tool responses, that are not currently evaluated. We argue that every boundary in an agentic pipeline is a site of potential privacy violation and must be assessed independently. To support this, we introduce the Privacy Flow Graph, a Contextual Integrity-grounded framework that decomposes agentic execution into a sequence of information flows, each annotated with the five CI parameters, and traces violations to their point of origin. We present AgentSCOPE, a benchmark of 62 multi-tool scenarios across eight regulatory domains with ground truth at every pipeline stage. Our evaluation across seven state-of-the-art LLMs show that privacy violations in the pipeline occur in over 80% of scenarios, even when final outputs appear clean (24%), with most violations arising at the tool-response stage where APIs return sensitive data indiscriminately. These results indicate that output-level evaluation alone substantially underestimates the privacy risk of agentic systems.
