BEAP-Agent: Backtrackable Execution and Adaptive Planning for GUI Agents

2026-01-29

AI Summary

This work addresses the limited error-recovery capability of existing GUI agents in long-horizon tasks, which often fail to recover once they deviate onto incorrect execution paths. To overcome this, the authors propose a cooperative framework based on depth-first search that integrates a planner, an executor, and a tracker, and introduces—for the first time—a systematic multi-level state backtracking mechanism. This mechanism enables dynamic task updates and adaptive planning, significantly enhancing the agent’s robustness and exploration efficiency in complex GUI environments. Evaluated on the OSWorld benchmark, the proposed approach achieves an accuracy of 28.2%, demonstrating its effectiveness in real-world GUI interaction scenarios.

Technology Category

• Multiagent Systems: Multiagent Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Heuristic Search

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

GUI agents are designed to automate repetitive tasks and enhance productivity. However, existing GUI agents struggle to recover once they follow an incorrect exploration path, often leading to task failure. In this work, we model GUI task execution as a DFS process and propose BEAP-Agent, a DFS-based framework that supports long-range, multi-level state backtracking with dynamic task tracking and updating. The framework consists of three collaborative components: Planner, Executor, and Tracker. Together, they enable effective task exploration and execution. BEAP-Agent fills the gap in systematic backtracking mechanisms for GUI agents, offering a systematic solution for long-horizon task exploration. We conducted a systematic evaluation on the OSWorld benchmark, where BEAP-Agent achieved an accuracy of 28.2%, validating the effectiveness of the proposed method.

Problem

• GUI agents
• task failure
• backtracking
• long-horizon tasks
• exploration recovery

Innovation

• Backtrackable Execution
• Adaptive Planning
• GUI Agents
• Depth-First Search
• State Backtracking