PersonalAlign: Hierarchical Implicit Intent Alignment for Personalized GUI Agent with Long-Term User-Centric Records

2026-01-14

AI Summary

Existing GUI agents struggle to align with users’ complex implicit intentions under ambiguous instructions and lack the capacity for personalized understanding and proactive assistance based on long-term behavioral patterns. This work proposes HIM-Agent, a hierarchical implicit intention alignment framework that formally defines this task for the first time. By structuring user preferences and routine behaviors, HIM-Agent enables the interpretation of underspecified intents and prediction of latent actions. We introduce AndroidIntent, the first benchmark supporting personalized evaluation, and design a continuously updated memory mechanism compatible with multimodal large language models such as GPT-5 and Qwen3-VL. Experimental results demonstrate that HIM-Agent improves execution accuracy by 15.7% and proactive suggestion performance by 7.3% on the AndroidIntent benchmark.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Modeling other Agents

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

While GUI agents have shown strong performance under explicit and completion instructions, real-world deployment requires aligning with users'more complex implicit intents. In this work, we highlight Hierarchical Implicit Intent Alignment for Personalized GUI Agent (PersonalAlign), a new agent task that requires agents to leverage long-term user records as persistent context to resolve omitted preferences in vague instructions and anticipate latent routines by user state for proactive assistance. To facilitate this study, we introduce AndroidIntent, a benchmark designed to evaluate agents'ability in resolving vague instructions and providing proactive suggestions through reasoning over long-term user records. We annotated 775 user-specific preferences and 215 routines from 20k long-term records across different users for evaluation. Furthermore, we introduce Hierarchical Intent Memory Agent (HIM-Agent), which maintains a continuously updating personal memory and hierarchically organizes user preferences and routines for personalization. Finally, we evaluate a range of GUI agents on AndroidIntent, including GPT-5, Qwen3-VL, and UI-TARS, further results show that HIM-Agent significantly improves both execution and proactive performance by 15.7% and 7.3%.

Problem

• Implicit Intent Alignment
• Personalized GUI Agent
• Long-Term User Records
• Proactive Assistance
• Vague Instructions

Innovation

• Hierarchical Implicit Intent Alignment
• Personalized GUI Agent
• Long-Term User Records
• Proactive Assistance
• AndroidIntent