SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems

2025-06-09

AI Summary

Current LLM/VLM-based autonomous agent systems suffer from uncontrolled information leakage, decision contamination, and concurrency-induced inconsistency—critical safety and reliability bottlenecks. This paper proposes SAFEOFLOW, the first trusted runtime protocol integrating fine-grained information flow control (IFC) with transactional execution to enable cross-agent/tool/environment data provenance and co-guarantee confidentiality and integrity. Key innovations include dynamically labeled inference constraints, write-ahead logging, security-aware rollback, and a multimodal agent coordination protocol. Evaluated on the SAFEOFLOWBENCH benchmark, SAFEOFLOW reduces decision contamination rate by 72% in high-risk scenarios and improves concurrent task success rate to 94.3%, significantly outperforming state-of-the-art approaches.

Technology Category

• Natural Language Processing: Safety and Robustness
• Multiagent Systems: Agent Communication
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Security and Privacy: Data transparency and provenance
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Recent advances in large language models (LLMs) and vision-language models (VLMs) have enabled powerful autonomous agents capable of complex reasoning and multi-modal tool use. Despite their growing capabilities, today's agent frameworks remain fragile, lacking principled mechanisms for secure information flow, reliability, and multi-agent coordination. In this work, we introduce SAFEFLOW, a new protocol-level framework for building trustworthy LLM/VLM-based agents. SAFEFLOW enforces fine-grained information flow control (IFC), precisely tracking provenance, integrity, and confidentiality of all the data exchanged between agents, tools, users, and environments. By constraining LLM reasoning to respect these security labels, SAFEFLOW prevents untrusted or adversarial inputs from contaminating high-integrity decisions. To ensure robustness in concurrent multi-agent settings, SAFEFLOW introduces transactional execution, conflict resolution, and secure scheduling over shared state, preserving global consistency across agents. We further introduce mechanisms, including write-ahead logging, rollback, and secure caches, that further enhance resilience against runtime errors and policy violations. To validate the performances, we built SAFEFLOWBENCH, a comprehensive benchmark suite designed to evaluate agent reliability under adversarial, noisy, and concurrent operational conditions. Extensive experiments demonstrate that agents built with SAFEFLOW maintain impressive task performance and security guarantees even in hostile environments, substantially outperforming state-of-the-art. Together, SAFEFLOW and SAFEFLOWBENCH lay the groundwork for principled, robust, and secure agent ecosystems, advancing the frontier of reliable autonomy.

Problem

• Ensures secure information flow in autonomous agent systems
• Provides transactional execution for multi-agent coordination
• Enhances resilience against runtime errors and policy violations

Innovation

• Enforces fine-grained information flow control
• Introduces transactional execution for multi-agent settings
• Enhances resilience with logging and rollback mechanisms