OpenCUA: Open Foundations for Computer-Use Agents

2025-08-12

AI Summary

Current computer-using agents (CUAs) rely on proprietary, closed systems, hindering open research and reproducibility. To address this, we introduce OpenCUA—the first open-source foundational framework for end-to-end task automation across operating systems and applications. Our method comprises three core components: (1) AgentNet, a large-scale, multi-platform task dataset covering Windows, macOS, and Linux, with annotations from 200+ applications; (2) a reflective, long-chain reasoning pipeline for robust data transformation; and (3) a vision-language model architecture integrating demonstration collection, state-action pair generation, and a scalable training framework. Evaluated on OSWorld-Verified, OpenCUA-32B achieves a new state-of-the-art average success rate of 34.8% among open models—substantially outperforming GPT-4o—and demonstrates strong cross-domain generalization and computational scalability.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Other Foundations of Multi Agent Systems

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Vision-language models have demonstrated impressive capabilities as computer-use agents (CUAs) capable of automating diverse computer tasks. As their commercial potential grows, critical details of the most capable CUA systems remain closed. As these agents will increasingly mediate digital interactions and execute consequential decisions on our behalf, the research community needs access to open CUA frameworks to study their capabilities, limitations, and risks. To bridge this gap, we propose OpenCUA, a comprehensive open-source framework for scaling CUA data and foundation models. Our framework consists of: (1) an annotation infrastructure that seamlessly captures human computer-use demonstrations; (2) AgentNet, the first large-scale computer-use task dataset spanning 3 operating systems and 200+ applications and websites; (3) a scalable pipeline that transforms demonstrations into state-action pairs with reflective long Chain-of-Thought reasoning that sustain robust performance gains as data scales. Our end-to-end agent models demonstrate strong performance across CUA benchmarks. In particular, OpenCUA-32B achieves an average success rate of 34.8% on OSWorld-Verified, establishing a new state-of-the-art (SOTA) among open-source models and surpassing OpenAI CUA (GPT-4o). Further analysis confirms that our approach generalizes well across domains and benefits significantly from increased test-time computation. We release our annotation tool, datasets, code, and models to build open foundations for further CUA research.

Problem

• OpenCUA addresses closed commercial CUA systems lacking transparency.
• It provides open frameworks to study CUA capabilities and risks.
• The solution includes scalable data, models, and annotation tools.

Innovation

• Open-source framework for vision-language CUA models
• Large-scale dataset spanning multiple OS and applications
• Scalable pipeline with reflective Chain-of-Thought reasoning