Safeguard-by-Development: A Privacy-Enhanced Development Paradigm for Multi-Agent Collaboration Systems

2025-05-07

AI Summary

Multi-agent collaborative systems (MACS) leveraging large language models (LLMs) face severe sensitive data leakage risks due to frequent inter-agent and agent–environment interactions (e.g., with LLMs, external tools, and users); existing systems lack fine-grained privacy control mechanisms. To address this, we propose Maris—the first privacy-enhancing paradigm designed at the system development level. Maris innovatively embeds a reference monitor deep within the core dialogue components of multi-agent systems, enabling policy-driven message interception and dynamic permission validation. It is fully integrated end-to-end into the AutoGen framework as a deployable module. Experimental evaluation across healthcare, supply chain, and personalized recommendation domains demonstrates that Maris blocks over 98% of simulated leakage paths while incurring less than 8% latency overhead. Maris thus achieves a strong balance among high security assurance, low performance cost, and practical engineering deployability.

Technology Category

• Multiagent Systems: Agent Communication
• Machine Learning: Privacy
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Multi-agent collaboration systems (MACS), powered by large language models (LLMs), solve complex problems efficiently by leveraging each agent's specialization and communication between agents. However, the inherent exchange of information between agents and their interaction with external environments, such as LLM, tools, and users, inevitably introduces significant risks of sensitive data leakage, including vulnerabilities to attacks like prompt injection and reconnaissance. Existing MACS fail to enable privacy controls, making it challenging to manage sensitive information securely. In this paper, we take the first step to address the MACS's data leakage threat at the system development level through a privacy-enhanced development paradigm, Maris. Maris enables rigorous message flow control within MACS by embedding reference monitors into key multi-agent conversation components. We implemented Maris as an integral part of AutoGen, a widely adopted open-source multi-agent development framework. Then, we evaluate Maris for its effectiveness and performance overhead on privacy-critical MACS use cases, including healthcare, supply chain optimization, and personalized recommendation system. The result shows that Maris achieves satisfactory effectiveness, performance overhead and practicability for adoption.

Problem

• Addressing sensitive data leakage risks in multi-agent collaboration systems
• Enhancing privacy controls in MACS development to prevent information exposure
• Mitigating vulnerabilities like prompt injection in LLM-powered agent interactions

Innovation

• Privacy-enhanced development paradigm for MACS
• Embedded reference monitors for message control
• Integration into AutoGen framework for practicality