Skill-Guided Continuation Distillation for GUI Agents

2026-06-17

AI Summary

This work addresses the challenge that GUI agents in closed-loop execution often encounter out-of-distribution states—caused by policy drift from expert trajectories—leading to decision failures due to insufficient supervision. To mitigate this, the paper proposes an iterative self-improvement framework in which an unskilled-guided policy actively explores such deviated states, while a skill-guided policy generates successful trajectory continuations. These continuations, combined with the original expert data, provide comprehensive supervision. The key innovation lies in introducing, for the first time, a skill-guided continuation distillation mechanism that extracts critical skills—such as continuation plans and target subtasks—from both successful and failed trajectories to fill supervision gaps. Evaluated on the OSWorld-Verified benchmark, the method significantly boosts success rates of three base models from approximately 30% to over 50%, demonstrating its effectiveness and generalizability.

Abstract

Improving GUI agents typically relies on behavior cloning on expert trajectories. However, as the current policy deviates from the expert policy, it inevitably encounters policy-induced off-trajectory states during closed-loop execution, i.e., states that fall outside the expert trajectories. Since expert trajectories provide no demonstrations for these unseen states, such states receive no effective supervision, leaving the policy unable to select the correct action. To close this supervision gap, we propose Skill-Guided Continuation Distillation (SGCD), an iterative self-improvement framework. SGCD first runs the plain policy without skill guidance for a few steps to reach realistic off-trajectory states. From these states, a skill-guided policy then completes the task and produces successful continuations, which are mixed with expert trajectories to supply supervision over policy-induced off-trajectory states. The skills are extracted from both successful and failed rollouts, consisting of Continuation Plans, Critical Targets, Failure Traps, and Success Criteria. On OSWorld-Verified, SGCD improves the success rate of three base models from the low-30\% range to over 50\%, demonstrating its effectiveness and generality.

Problem

• GUI agents
• off-trajectory states
• behavior cloning
• supervision gap
• expert trajectories

Innovation

• Skill-Guided Continuation Distillation
• off-trajectory states
• GUI agents
• self-improvement framework
• behavior cloning