SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL

2026-07-13

AI Summary

This work addresses the limitations faced by existing computer-using agents, which are hindered by the scarcity of verifiable data and the inefficiency of online reinforcement learning. To overcome these challenges, the authors propose a unified framework that synthesizes high-fidelity tasks through a VeriGen task generation mechanism, enhanced by a Frontier Sampling strategy and a Visual Context Segmentation training approach to substantially scale up data volume and improve training efficiency. The system integrates key innovations including a multi-agent feedback loop, Docker-based concurrent interaction, and a visual context sliding window. Evaluated on the OSWorld and ScienceBoard benchmarks, the method achieves state-of-the-art success rates of 68.7% and 54.0%, respectively, establishing a new performance record for open-source computer-using agents.

Abstract

Computer use agents (CUAs) are emerging as a powerful interface for automating complex digital workflows through visual perception and GUI execution. Online reinforcement learning with verifiable rewards (RLVR) has emerged as a key direction for scaling their capabilities. However, this paradigm is bottlenecked by verifiable data scarcity and online RL inefficiency. To break these barriers, we introduce ScaleCUA, a unified framework that scales online RL for CUAs via verifiable task synthesis and efficient training. At the data level, we design VeriGen, an end-to-end framework for generating verifiable RL tasks through iterative docker interactions and a multi-agent feedback loop. Scaled to 100+ concurrent agent workers via a shared docker interaction probe, this pipeline produces 24K+ verifiable tasks and nearly 3K high-quality RL tasks. To maximize sample efficiency, we propose Frontier Sampling, which tracks per-task capability and allocates rollouts to the current learning frontier. On the training side, we further design Visual Context Segmentation, a sliding window over recent visual context that balances rollout and training-engine pressure, yielding a 2.83x training speedup over step-wise decomposition. Together, ScaleCUA achieves 68.7% on OSWorld and 54.0% on ScienceBoard, establishing new state-of-the-art performance among open-source computer use agents. Code, models, and datasets are available at https://github.com/THUDM/SCALE-CUA.

Problem

• computer use agents
• verifiable data scarcity
• online reinforcement learning inefficiency
• task synthesis
• sample efficiency

Innovation

• Verifiable Task Synthesis
• Efficient Online RL
• Frontier Sampling
• Visual Context Segmentation
• Computer Use Agents