Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization

2026-09-02

AI Summary

本文通过系统角度研究了GUI代理的效率问题，包括观察、记忆、行动和运行时优化，并总结了提高效率的主要方法。

Abstract

GUI agents increasingly operate across websites, mobile apps, and desktop environments, yet the field still reports progress primarily through task success. We argue that practical deployment depends equally on efficiency: how much context, computation, action budget, and runtime overhead an agent consumes while succeeding. This survey studies efficient GUI agents through an end-to-end systems lens that preserves the current technical axes of observation efficiency, context and memory efficiency, action efficiency, and planner-side/system efficiency. For each subsection, we expand the seed literature through targeted search plus backward and forward citation chaining, then synthesize the dominant mechanisms, reported efficiency signals, and new overheads they introduce. Across the literature, recent progress converges on a small set of recurring ideas: selective reading instead of full-context ingestion, global-to-local visual allocation, recoverable memory rather than raw history replay, verification-aware control, and hybrid runtimes that can switch between GUI and non-GUI execution. We conclude by identifying the main open problems, including honest accounting of verifier cost, cross-benchmark comparability, and co-design of observation, memory, and execution layers under real latency and privacy constraints.

Problem

• GUI agents
• efficiency
• context
• computation
• runtime overhead

Innovation

• selective reading
• global-to-local visual allocation
• recoverable memory
• verification-aware control
• hybrid runtimes