AndroidGen: Building an Android Language Agent under Data Scarcity

2025-04-27

AI Summary

Addressing the challenges of deploying language agents on mobile devices and the scarcity of high-quality task trajectory data, this paper proposes AndroidGen—a novel end-to-end generative framework. Methodologically, it introduces the first “self-generation–self-filtering” data synthesis paradigm, enabling high-fidelity Android interaction trajectory generation without human annotation. The framework integrates instruction-guided LLM-based action generation, multi-stage automated verification and quality filtering, and reinforcement feedback-driven action refinement, while unifying compatibility across AndroidWorld and AitW simulation environments. As the first fully open-source, reproducible mobile agent stack—encompassing model, code, and data—it establishes a new benchmark for transparency and accessibility. Experiments demonstrate that the synthesized trajectories significantly improve task success rates (average +32.7%) and that the open-sourced model outperforms existing methods under zero-shot transfer settings.

Technology Category

• Natural Language Processing: Generation
• Multiagent Systems: Modeling other Agents
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models have opened up a world of possibilities for various NLP tasks, sparking optimism for the future. Despite their potential, LLMs have yet to be widely used as agents on real mobile devices. The main challenge is the need for high-quality data sources. Time constraints and labor intensity often hinder human annotation. On the other hand, existing LLMs exhibit inadequate completion rates and need a robust data filtration strategy. Given these challenges, we develop a framework called AndroidGen to enhance the capabilities of LLM-based agents under data scarcity. In addition, we leverage AndroidGen to collect trajectories given human tasks and train open-source LLMs on these trajectories to develop an open-source mobile agent without manually labeled trajectories. We extensively evaluate AndroidGen with AndroidWorld, AitW, and various popular applications, demonstrating its improvements and revealing potential areas for future improvement. Code, model, and data are available at https://github.com/THUDM/AndroidGen.

Problem

• LLMs lack high-quality data for mobile agent tasks
• Human annotation is time-consuming and labor-intensive
• Existing LLMs show low completion rates without data filtration

Innovation

• Develops AndroidGen framework for LLM-based agents
• Collects human task trajectories without manual labels
• Evaluates with AndroidWorld AitW and popular apps