AIR: Improving Agent Safety through Incident Response

2026-02-12

AI Summary

This work addresses a critical gap in the safety mechanisms of large language model (LLM) agents, which predominantly focus on preventive measures and lack capabilities for responding to and recovering from incidents after they occur. To bridge this gap, the authors propose AIR—the first incident response framework tailored for LLM agents—that integrates semantic anomaly detection, tool-driven containment, and recovery mechanisms within the agent’s execution loop. AIR also leverages a domain-specific language to automatically generate protective rules, enabling autonomous management across the full incident lifecycle. Experimental evaluation demonstrates that AIR achieves over 90% success rates in detection, remediation, and eradication across three representative agent types, with automatically generated rules performing comparably to handcrafted ones, all while maintaining acceptable runtime overhead.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Multiagent Systems: Agent Communication

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Large Language Model (LLM) agents are increasingly deployed in practice across a wide range of autonomous applications. Yet current safety mechanisms for LLM agents focus almost exclusively on preventing failures in advance, providing limited capabilities for responding to, containing, or recovering from incidents after they inevitably arise. In this work, we introduce AIR, the first incident response framework for LLM agent systems. AIR defines a domain-specific language for managing the incident response lifecycle autonomously in LLM agent systems, and integrates it into the agent's execution loop to (1) detect incidents via semantic checks grounded in the current environment state and recent context, (2) guide the agent to execute containment and recovery actions via its tools, and (3) synthesize guardrail rules during eradication to block similar incidents in future executions. We evaluate AIR on three representative agent types. Results show that AIR achieves detection, remediation, and eradication success rates all exceeding 90%. Extensive experiments further confirm the necessity of AIR's key design components, show the timeliness and moderate overhead of AIR, and demonstrate that LLM-generated rules can approach the effectiveness of developer-authored rules across domains. These results show that incident response is both feasible and essential as a first-class mechanism for improving agent safety.

Problem

• LLM agent safety
• incident response
• autonomous systems
• safety mechanisms
• failure recovery

Innovation

• Incident Response
• LLM Agents
• Autonomous Safety
• Guardrail Synthesis
• Domain-Specific Language