PalmClaw: A Native On-Device Agent Framework for Mobile Phones

2026-07-14

AI Summary

Existing mobile agents rely heavily on GUI-based interactions, suffering from verbose action sequences, tight coupling with interface layouts, inability to directly invoke device capabilities, and ambiguous execution boundaries. This work proposes the first native on-device agent framework that encapsulates device functionalities as tools with well-defined parameters, structured outputs, and clear execution scopes. The framework unifies session management, memory, skills, and the agent loop entirely on the device. By bypassing GUI constraints, the approach achieves substantial performance gains: it improves task success rate by 11.5% relative to the strongest baseline, reduces task completion time by 94.9%, lowers deployment overhead, and enables explicit, controllable operation execution.

Abstract

Large Language Model (LLM) agents have moved beyond generating responses to executing multi-step tasks by calling tools, observing the results, and iteratively deciding the next action. Most agent systems run on desktops or servers, which support tool use and task automation. Mobile devices are also important agent environments because they are widely accessible and contain users' data, sensors, and daily-use applications. Existing mobile agents mainly operate smartphones through graphical user interface (GUI) actions such as tapping, swiping, and typing, which often form long, interface-dependent sequences, cannot directly access device capabilities, and make execution boundaries difficult to define. We present \textbf{PalmClaw}, an open-source agent framework that runs natively on mobile phones and manages the sessions, memory, skills, tools, and agent loop directly on the device. PalmClaw exposes device capabilities as device tools with explicit arguments, structured results, and clearly defined execution boundaries. This design enables agents to use mobile capabilities directly while keeping each action explicit and controlled. Experiments show an 11.5\% relative improvement in task success and a 94.9\% reduction in completion time over the strongest baseline, with lower setup burden and traces illustrating how execution boundaries are applied. Code is available at https://github.com/ModalityDance/PalmClaw.

Problem

• mobile agents
• LLM agents
• device capabilities
• execution boundaries
• GUI actions

Innovation

• on-device agent
• mobile LLM agent
• device tools
• execution boundaries
• native mobile framework