PAC-BENCH: Evaluating Multi-Agent Collaboration under Privacy Constraints

2026-04-13

AI Summary

This study addresses the lack of systematic understanding regarding multi-agent coordination mechanisms under privacy constraints and their impact on collaborative performance. The work proposes PAC-BENCH, a novel benchmark that, for the first time, enables systematic evaluation of how privacy limitations affect collaboration through a standardized multi-agent simulation environment integrated with behavioral analysis and coordination metrics. The investigation uncovers three distinct coordination failure modes induced by privacy: premature privacy leakage, over-conservative abstraction, and privacy-induced hallucination. Findings demonstrate a significant degradation in collaborative performance under privacy constraints, with outcomes becoming disproportionately dependent on the initiating agent’s behavior. These results underscore the urgent need to develop new coordination mechanisms that go beyond current capabilities to effectively operate within privacy-preserving settings.

Technology Category

• Multiagent Systems: Coordination and Collaboration
• Game Theory and Economic Paradigms: Coordination and Collaboration
• Machine Learning: Privacy

Application Category

• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

We are entering an era in which individuals and organizations increasingly deploy dedicated AI agents that interact and collaborate with other agents. However, the dynamics of multi-agent collaboration under privacy constraints remain poorly understood. In this work, we present $PAC\text{-}Bench$, a benchmark for systematic evaluation of multi-agent collaboration under privacy constraints. Experiments on $PAC\text{-}Bench$ show that privacy constraints substantially degrade collaboration performance and make outcomes depend more on the initiating agent than the partner. Further analysis reveals that this degradation is driven by recurring coordination breakdowns, including early-stage privacy violations, overly conservative abstraction, and privacy-induced hallucinations. Together, our findings identify privacy-aware multi-agent collaboration as a distinct and unresolved challenge that requires new coordination mechanisms beyond existing agent capabilities.

Problem

• multi-agent collaboration
• privacy constraints
• coordination breakdowns
• privacy-aware AI
• agent benchmarking

Innovation

• multi-agent collaboration
• privacy constraints
• benchmark
• coordination breakdown
• privacy-aware AI