Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents

2026-06-30

AI Summary

This work addresses a critical limitation in existing agent self-improvement methods, which typically leverage only successful trajectories while discarding informative failure cases. To bridge this gap, the authors propose a failure-oriented, inference-time self-improvement mechanism that systematically converts failed trajectories into actionable refinement signals. Their approach employs a multimodal large language model to diagnose the root causes of failures and generate lightweight code patches, which—after minimal human validation—are dynamically integrated to update the agent’s reasoning strategy. Notably, this method incurs no additional training costs and introduces only modest inference overhead. Evaluated on the OSWorld benchmark, it boosts the success rate of OpenCUA-72B from 42.3% to 48.9%, yielding a +6.6 percentage point improvement.

Abstract

Computer-use agents, which leverage multimodal large language models (MLLMs) to operate computers and complete tasks, have attracted significant attention for their utility and versatility. A major challenge in developing these agents is collecting large-scale, high-quality trajectories. The standard approach generates synthetic data through a self-improving loop: an agent is placed in a verifiable environment and iteratively fine-tuned on its successful trajectories. Despite its effectiveness, this paradigm exploits only successful trajectories and discards the failed ones, even though failures carry rich information about a model's weaknesses. In this work, we explore a complementary failure-driven self-improvement loop, a data-centric paradigm that turns failed trajectories into agent improvements. Specifically, we employ an LLM to diagnose failure modes, propose inference-time solutions, and generate code patches -- lightly verified by humans -- that upgrade the agent. We validate this approach with the state-of-the-art OpenCUA-72B model on the OSWorld benchmark, improving the success rate from 42.3% to 48.9%, a gain of 6.6 percentage points, without any additional training cost and with only modest inference overhead. Our results demonstrate that failure-driven self-improvement is a viable complement to success-based pipelines, enabling more efficient agent improvement.

Problem

• computer-use agents
• failure-driven learning
• self-improvement
• multimodal large language models
• trajectory data

Innovation

• failure-driven learning
• inference-time self-improvement
• computer-use agents
• code patching
• multimodal LLMs