STEVE: AStep Verification Pipeline for Computer-use Agent Training

2025-03-16

AI Summary

To address the scarcity of high-quality trajectory data and the poor scalability of behavior cloning in training GUI-operating AI agents, this paper proposes a lightweight, efficient training paradigm for desktop environments. Methodologically: (1) we construct a large-scale instruction-driven framework for collecting suboptimal trajectories; (2) we introduce a novel step-level automated verification pipeline leveraging GPT-4o’s multimodal reasoning to perform binary correctness assessment of each action conditioned on pre- and post-action screen states; (3) we integrate prospect theory to model action-value biases, enabling joint optimization of positive and negative samples; (4) we design a 7B-parameter vision-language model (VLM) fine-tuning strategy tailored for GUI agents, incorporating frame-difference analysis to enhance state perception efficiency. Evaluated on the realistic WinAgentArena desktop benchmark, our approach achieves state-of-the-art performance with significantly reduced training cost—marking the first successful deployment of a 7B-scale VLM for efficient end-to-end GUI control in complex tasks.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Adversarial Agents
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Developing AI agents to autonomously manipulate graphical user interfaces is a long challenging task. Recent advances in data scaling law inspire us to train computer-use agents with a scaled instruction set, yet using behavior cloning to train agents still requires immense high-quality trajectories. To meet the scalability need, we designed STEVE, a step verification pipeline for computer-use agent training. First, we establish a large instruction set for computer-use agents and collect trajectory data with some suboptimal agents. GPT-4o is used to verify the correctness of each step in the trajectories based on the screens before and after the action execution, assigning each step with a binary label. Last, we adopt the Kahneman and Tversky Optimization to optimize the agent from the binary stepwise labels. Extensive experiments manifest that our agent outperforms supervised finetuning by leveraging both positive and negative actions within a trajectory. Also, STEVE enables us to train a 7B vision-language model as a computer-use agent, achieving leading performance in the challenging live desktop environment WinAgentArena with great efficiency at a reduced cost. Code and data: https://github.com/FanbinLu/STEVE.

Problem

• Develop AI agents for GUI manipulation autonomously.
• Train agents using scaled instruction sets and behavior cloning.
• Optimize agent performance with step verification and binary labels.

Innovation

• Large instruction set for agent training
• GPT-4o verifies trajectory step correctness
• Kahneman-Tversky Optimization for agent improvement