RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems

2026-06-22

AI Summary

Existing security evaluation methods struggle to conduct unified and dynamic red-teaming assessments of heterogeneous AI agent systems due to the absence of a cross-architecture general framework. This work proposes RIFT-Bench, a dynamic red-teaming approach based on hierarchical graph representations, which enables unified modeling, multi-target adversarial attacks, and comprehensive evaluation through a two-stage pipeline comprising automated structural discovery and adaptive adversarial probing. RIFT-Bench introduces, for the first time, a graph-based representation tailored for AI agents alongside a dynamic probing mechanism, facilitating cross-architectural evaluation and validation of mitigation strategies. Experiments across 45 heterogeneous agent systems demonstrate that RIFT-Bench achieves strong generalization and scalability, establishing a robust foundation for the security assessment of AI agent systems.

Abstract

Agentic AI systems powered by large language models (LLMs) are rapidly evolving into autonomous decision-making systems, exposing attack vectors beyond those of traditional LLM vulnerabilities. Existing security evaluations are often tied to specific implementations or domains, limiting unified comparison across heterogeneous systems. To address this gap, we introduce RIFT-Bench, a graph representation-driven methodology for dynamic red-teaming that enables unified evaluations across diverse agentic architectures. Building on a novel hierarchical representation, RIFT-Bench operates in two automated phases: Discovery, which extracts system structure, and Scanning, which deploys adaptive adversarial attacks and produces a comprehensive evaluation report. It evaluates the examined system itself, leveraging a broad set of dynamically adaptable adversarial probes across diverse attack vectors and objectives. We demonstrate the effectiveness of the proposed evaluation pipeline across 45 agentic systems spanning a diverse range of implementations, showing that the approach generalizes effectively to heterogeneous agentic architectures. Beyond systems and attacks, RIFT-Bench also supports direct evaluation of mitigation strategies. These key capabilities make RIFT-Bench a scalable foundation for security evaluation of agentic AI systems.

Problem

• Agentic AI
• Security Evaluation
• Red-teaming
• LLM Vulnerabilities
• Heterogeneous Systems

Innovation

• dynamic red-teaming
• agentic AI systems
• graph representation
• hierarchical representation
• adversarial evaluation