Search papers, labs, and topics across Lattice.
This paper introduces a simulation platform for evaluating LLM agent privacy in multi-agent social environments over extended periods. They find that privacy violations are significantly amplified in multi-turn social interactions compared to single-turn evaluations, with leakage rates increasing from 19.95% to 45.30% across OpenAI models. Furthermore, the study reveals that privacy breaches are socially contagious, and while privacy instructions offer some mitigation, they don't eliminate the risk, highlighting the limitations of current safety benchmarks.
LLM agents spill secrets like gossip after a few rounds, with privacy violations more than doubling in social settings compared to isolated chats.
LLM safety evaluations predominantly test models in isolation, yet deployed AI agents increasingly operate within persistent social environments alongside other agents. We introduce a Moltbook-style simulation platform where thousands of LLM agents interact across communities over a simulated month, and use it to evaluate privacy as a downstream safety concern under varying degrees of social pressure. We find that shifting from single turn to multi turn social evaluation amplifies privacy violations (CIMemories 19.95% to Ours 45.30% across OpenAI models), that leakage is socially contagious, with agents 8 times more likely to disclose sensitive information after observing a peer do so, and that explicit privacy instructions reduce but do not eliminate this effect, leaving leakage rates above 37.8% even with safeguards. Our findings suggest that static chat based safety benchmarks systematically underestimate risks in agentic deployment, and that social context alone is sufficient to elicit sensitive disclosures that single turn evaluations would never surface.
