Search papers, labs, and topics across Lattice.
This paper introduces PiSAs, a novel benchmark designed to evaluate privacy risks in multi-user agentic systems by focusing on contextual integrity (CI) in shared environments. The study reveals that while system design can enhance CI compliance, the performance of state-of-the-art LLMs is hindered by their inability to consistently filter inappropriate content and restrict access to authorized users. The findings highlight significant data spillage risks that existing benchmarks fail to address, emphasizing the urgent need for improved privacy-preserving strategies in AI systems.
Even advanced LLMs struggle to prevent privacy breaches in multi-user settings, exposing critical data spillage risks that current benchmarks overlook.
As LLM agents evolve from single-user assistants into shared organizational infrastructure, new privacy risks emerge: inappropriate information may not only be exposed through outputs for external recipients, but also internally across users through inter-agent messages, shared memory and agents. These data spillage risks are not captured by existing privacy benchmarks grounded in contextual integrity (CI) as they focus primarily on either single-user settings or interactions between independently owned agents. We introducePiSAs (Privacy in Shared Agentic systems), a benchmark for assessing unintentional leaks with dual CI annotations: whether an information is appropriate for the task, and which users may legitimately access it. This enables direct measurement of cross-user spillage across agentic system components and interfaces, such as outputs, inter-agent communication, and memory. PiSAsis system-agnostic and supports evaluation across different agent topologies and memory regimes. We find that, although system design improves CI compliance, results are bottlenecked by incorrect LLM judgment calls: even state-of-the-art models fail to reliably filter inappropriate content or restrict transmission to authorized users. Our findings underscore the need for privacy-preserving strategies, beyond those studied in this work.
Mila