What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving States

2026-06-30

AI Summary

Existing mobile GUI agents struggle with long-horizon tasks due to passive memory mechanisms that fail to effectively track the roles and states of task-relevant data, often resulting in redundant or omitted actions. To address this limitation, this work proposes Active Task-driven Memory (ATMem), which models memory as an active execution state explicitly linked to task roles and states. Integrated within the STR-GRPO online reinforcement learning framework, ATMem incorporates a memory cost-aware reward mechanism and a contrastive memory utilization strategy to dynamically optimize memory retrieval. Evaluated on a newly constructed, challenging benchmark for long-horizon mobile tasks, the proposed approach significantly improves task completion accuracy and robustness, enabling precise handling of compliant actions while reliably rejecting invalid operations.

Abstract

Mobile GUI agents increasingly face long-horizon tasks that require reading, updating, and reusing task-relevant data across pages and applications. Existing memory methods treat memory largely as passive storage, where past observations are accumulated and retrieved when needed. Yet retrieving a value does not reveal its current role in the workflow. The agent must still infer from accumulated records whether the value should be used now, has already been used, or must wait for a later dependency. This implicit reconstruction becomes unreliable in long trajectories with similar fields, repeated values, distractors, and outdated states, causing repeated or missed operations. We propose Active Task Driving Memory (ATMem), which shifts GUI-agent memory from passive storage to an actively maintained execution state. ATMem maintains task-relevant information as a continually updated execution state that links each value to its role and current status, enabling action selection based on the current workflow state. We therefore introduce \textbf{STR-GRPO}, an online reinforcement learning method that learns to use ATMem selectively according to its contribution to task completion. STR-GRPO contrasts memory-on and memory-off rollouts to estimate when memory use improves execution, while memory-cost-aware reward discourages costly memory usage that does not improve execution. To evaluate whether agents can complete all in-scope work while avoiding out-of-scope actions over long-horizon execution, we build a challenging mobile benchmark. From a list of near identical entries, agents must act on every entry that satisfies the instruction and reject entries that violate its constraints.

Problem

• GUI agents
• long-horizon tasks
• memory management
• task execution state
• mobile automation

Innovation

• Active Task Driving Memory
• STR-GRPO
• GUI agents
• memory-aware reinforcement learning
• long-horizon task execution