PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems

2026-07-06

AI Summary

This work addresses the critical challenge of cross-user privacy leakage in multi-user shared agent systems, where shared memory and internal communication channels can inadvertently expose sensitive information. Existing privacy evaluation methods based on contextual integrity offer insufficient coverage for such complex interactions. To bridge this gap, the paper introduces PiSAs, the first privacy assessment benchmark tailored to these systems. PiSAs employs dual annotations—task appropriateness and user access legitimacy—to quantitatively measure unintended information disclosure across multiple interfaces, including outputs, communications, and memory. The framework is system-agnostic, accommodating diverse agent topologies and memory mechanisms. Experimental results reveal that despite improved compliance in current designs, large language models still struggle to accurately discern contextual appropriateness and permission boundaries, leading advanced models to fail reliably in filtering inappropriate content or preventing unauthorized data transmission.

Abstract

As LLM agents evolve from single-user assistants into shared organizational infrastructure, new privacy risks emerge: inappropriate information may not only be exposed through outputs for external recipients, but also internally across users through inter-agent messages, shared memory and agents. These data spillage risks are not captured by existing privacy benchmarks grounded in contextual integrity (CI) as they focus primarily on either single-user settings or interactions between independently owned agents. We introducePiSAs (Privacy in Shared Agentic systems), a benchmark for assessing unintentional leaks with dual CI annotations: whether an information is appropriate for the task, and which users may legitimately access it. This enables direct measurement of cross-user spillage across agentic system components and interfaces, such as outputs, inter-agent communication, and memory. PiSAsis system-agnostic and supports evaluation across different agent topologies and memory regimes. We find that, although system design improves CI compliance, results are bottlenecked by incorrect LLM judgment calls: even state-of-the-art models fail to reliably filter inappropriate content or restrict transmission to authorized users. Our findings underscore the need for privacy-preserving strategies, beyond those studied in this work.

Problem

• contextual integrity
• multi-user agentic systems
• privacy leakage
• data spillage
• shared memory

Innovation

• Contextual Integrity
• Multi-User Agentic Systems
• Privacy Benchmark
• Information Spillage
• LLM Privacy Failures