RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments

2025-05-28

AI Summary

Computer-Using Agents (CUAs) operating in cross-OS and web-hybrid environments are vulnerable to indirect prompt injection attacks, yet existing evaluations lack realistic, controllable testbeds. Method: This paper introduces the first adversarial testing framework supporting coordinated Web-OS interaction, built upon a novel VM+Docker hybrid sandbox enabling direct attack-point initialization, decoupled navigation, and flexible scenario configuration. It releases RTC-Bench, the first end-to-end benchmark with 864 real-world tasks, alongside an adversarial scenario injection engine and a CUA behavioral quantification toolkit. Results: Experiments reveal alarmingly high attack success rates—up to 48% for state-of-the-art CUAs (e.g., Claude 4 Opus) and 7.6% even for the most robust (Operator); 92.5% of malicious tasks were fully executed, confirming severe real-world risks. This work establishes a new benchmark and methodology for rigorous CUA security evaluation.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Multiagent Systems: Adversarial Agents
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Computer-use agents (CUAs) promise to automate complex tasks across operating systems (OS) and the web, but remain vulnerable to indirect prompt injection. Current evaluations of this threat either lack support realistic but controlled environments or ignore hybrid web-OS attack scenarios involving both interfaces. To address this, we propose RedTeamCUA, an adversarial testing framework featuring a novel hybrid sandbox that integrates a VM-based OS environment with Docker-based web platforms. Our sandbox supports key features tailored for red teaming, such as flexible adversarial scenario configuration, and a setting that decouples adversarial evaluation from navigational limitations of CUAs by initializing tests directly at the point of an adversarial injection. Using RedTeamCUA, we develop RTC-Bench, a comprehensive benchmark with 864 examples that investigate realistic, hybrid web-OS attack scenarios and fundamental security vulnerabilities. Benchmarking current frontier CUAs identifies significant vulnerabilities: Claude 3.7 Sonnet | CUA demonstrates an ASR of 42.9%, while Operator, the most secure CUA evaluated, still exhibits an ASR of 7.6%. Notably, CUAs often attempt to execute adversarial tasks with an Attempt Rate as high as 92.5%, although failing to complete them due to capability limitations. Nevertheless, we observe concerning ASRs of up to 50% in realistic end-to-end settings, with the recently released frontier Claude 4 Opus | CUA showing an alarming ASR of 48%, demonstrating that indirect prompt injection presents tangible risks for even advanced CUAs despite their capabilities and safeguards. Overall, RedTeamCUA provides an essential framework for advancing realistic, controlled, and systematic analysis of CUA vulnerabilities, highlighting the urgent need for robust defenses to indirect prompt injection prior to real-world deployment.

Problem

• Testing computer-use agents for indirect prompt injection vulnerabilities
• Evaluating hybrid web-OS attack scenarios in realistic environments
• Assessing security risks of advanced CUAs in adversarial settings

Innovation

• Hybrid sandbox integrates VM and Docker
• Decouples adversarial evaluation from navigation limits
• Comprehensive benchmark with 864 attack scenarios