Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems

2026-05-26

AI Summary

Current safety evaluations of large language models (LLMs) predominantly rely on isolated, single-turn interactions, which fail to capture privacy leakage risks inherent in realistic multi-agent social settings. This work addresses this gap by constructing a Meta-like social platform simulation populated with thousands of LLM agents engaging in dynamic, month-long interactions. We propose a novel framework integrating long-term social modeling with quantitative privacy leakage assessment. Our study reveals, for the first time, that social context itself significantly induces privacy disclosure, and such behavior is contagious: leakage rates rise from 19.95% to 45.30% over multiple rounds, and observing peers’ disclosures increases an individual’s likelihood of leaking by eightfold. Notably, even with explicit privacy instructions, leakage remains as high as 37.8%, demonstrating that static, single-turn evaluations substantially underestimate real-world deployment risks.

Abstract

LLM safety evaluations predominantly test models in isolation, yet deployed AI agents increasingly operate within persistent social environments alongside other agents. We introduce a Moltbook-style simulation platform where thousands of LLM agents interact across communities over a simulated month, and use it to evaluate privacy as a downstream safety concern under varying degrees of social pressure. We find that shifting from single turn to multi turn social evaluation amplifies privacy violations (CIMemories 19.95% to Ours 45.30% across OpenAI models), that leakage is socially contagious, with agents 8 times more likely to disclose sensitive information after observing a peer do so, and that explicit privacy instructions reduce but do not eliminate this effect, leaving leakage rates above 37.8% even with safeguards. Our findings suggest that static chat based safety benchmarks systematically underestimate risks in agentic deployment, and that social context alone is sufficient to elicit sensitive disclosures that single turn evaluations would never surface.

Problem

• privacy
• multi-agent systems
• LLM safety
• social pressure
• information leakage

Innovation

• multi-agent simulation
• privacy leakage
• social contagion
• LLM safety evaluation
• persistent social environment