Terrarium: Revisiting the Blackboard for Multi-Agent Safety, Privacy, and Security Studies

2025-10-16

AI Summary

LLM-driven multi-agent systems (MAS) face critical security and privacy risks—including goal misalignment, malicious agent infiltration, communication hijacking, and data poisoning—particularly in automated collaborative tasks such as meeting scheduling. Method: This paper introduces the first modular, blackboard-based security testing framework for MAS, enabling fine-grained risk modeling, configurable collaboration protocols, and simulation of four distinct attack classes, integrated with dynamic access control and information-sharing mechanisms. Contribution/Results: Evaluated across three real-world collaborative scenarios, the framework significantly accelerates the prototyping, evaluation, and iterative refinement of defensive strategies. Its core innovation lies in re-purposing the classical blackboard architecture as a trusted experimental foundation for secure MAS, establishing a reproducible, scalable benchmark platform to advance robustness and controllability research for LLM-based MAS.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

A multi-agent system (MAS) powered by large language models (LLMs) can automate tedious user tasks such as meeting scheduling that requires inter-agent collaboration. LLMs enable nuanced protocols that account for unstructured private data, user constraints, and preferences. However, this design introduces new risks, including misalignment and attacks by malicious parties that compromise agents or steal user data. In this paper, we propose the Terrarium framework for fine-grained study on safety, privacy, and security in LLM-based MAS. We repurpose the blackboard design, an early approach in multi-agent systems, to create a modular, configurable testbed for multi-agent collaboration. We identify key attack vectors such as misalignment, malicious agents, compromised communication, and data poisoning. We implement three collaborative MAS scenarios with four representative attacks to demonstrate the framework's flexibility. By providing tools to rapidly prototype, evaluate, and iterate on defenses and designs, Terrarium aims to accelerate progress toward trustworthy multi-agent systems.

Problem

• Studying safety risks in multi-agent systems using LLMs
• Addressing privacy threats from malicious agents and attacks
• Developing security frameworks against data poisoning and misalignment

Innovation

• Repurposed blackboard design for modular testbed
• Implemented configurable multi-agent collaboration scenarios
• Provided tools for rapid defense prototyping