Formalizing the Safety, Security, and Functional Properties of Agentic AI Systems

2025-10-15

AI Summary

Fragmentation of communication protocols (e.g., MCP, A2A) in multi-agent AI systems introduces a semantic gap that impedes systematic, formal verification of safety and functional properties. Method: We propose the first domain-agnostic, rigorously formalized unified semantic modeling framework, integrating a host-agent model with a task lifecycle model to enable fine-grained modeling and verification of task-state transitions, error handling, deadlock, and security vulnerabilities. Contribution/Results: The framework formally defines 31 critical system properties, enabling cross-protocol consistency verification and bridging the semantic gap between task coordination and security assurance. Experimental evaluation demonstrates its effectiveness in detecting anomalous collaboration boundaries and significantly improving system reliability and robustness—particularly in high-risk scenarios.

Technology Category

• Multiagent Systems: Agent Communication
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Agentic AI systems, which leverage multiple autonomous agents and Large Language Models (LLMs), are increasingly used to address complex, multi-step tasks. The safety, security, and functionality of these systems are critical, especially in high-stakes applications. However, the current ecosystem of inter-agent communication is fragmented, with protocols such as the Model Context Protocol (MCP) for tool access and the Agent-to-Agent (A2A) protocol for coordination being analyzed in isolation. This fragmentation creates a semantic gap that prevents the rigorous analysis of system properties and introduces risks such as architectural misalignment and exploitable coordination issues. To address these challenges, we introduce a modeling framework for agentic AI systems composed of two foundational models. The first, the host agent model, formalizes the top-level entity that interacts with the user, decomposes tasks, and orchestrates their execution by leveraging external agents and tools. The second, the task lifecycle model, details the states and transitions of individual sub-tasks from creation to completion, providing a fine-grained view of task management and error handling. Together, these models provide a unified semantic framework for reasoning about the behavior of multi-AI agent systems. Grounded in this framework, we define 17 properties for the host agent and 14 for the task lifecycle, categorized into liveness, safety, completeness, and fairness. Expressed in temporal logic, these properties enable formal verification of system behavior, detection of coordination edge cases, and prevention of deadlocks and security vulnerabilities. Through this effort, we introduce the first rigorously grounded, domain-agnostic framework for the systematic analysis, design, and deployment of correct, reliable, and robust agentic AI systems.

Problem

• Formalizing safety and security properties of multi-agent AI systems
• Addressing fragmented communication protocols between autonomous agents
• Providing unified semantic framework for system behavior verification

Innovation

• A modeling framework formalizes host agent and task lifecycle
• Defines properties for formal verification using temporal logic
• Provides unified semantic framework for multi-agent systems