Zero-Permission Manipulation: Can We Trust Large Multimodal Model Powered GUI Agents?

2026-01-18

AI Summary

This work challenges the “visual atomicity” assumption underpinning large multimodal GUI agents on Android—namely, that UI states remain unchanged between observation and action execution—and demonstrates how this assumption can be exploited by malicious applications. The authors introduce Action Rebinding, a novel attack leveraging foreground activity switching and task resumption mechanisms to hijack agent actions during the observation-execution gap, bypassing user confirmation prompts without requiring any dangerous permissions. To enhance attack efficacy, they propose the Intent Alignment Strategy (IAS), which aligns malicious intent with legitimate user flows. Experimental results show 100% success in single-step action rebinding across six state-of-the-art GUI agents, reliable reproduction of multi-step attack chains, and an increase in verification bypass success from 0% to 100% with IAS. Notably, the malicious apps implementing these techniques remain undetected by VirusTotal and similar platforms.

Technology Category

• Multiagent Systems: Adversarial Agents
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Intelligent User Interfaces

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Large multimodal model powered GUI agents are emerging as high-privilege operators on mobile platforms, entrusted with perceiving screen content and injecting inputs. However, their design operates under the implicit assumption of Visual Atomicity: that the UI state remains invariant between observation and action. We demonstrate that this assumption is fundamentally invalid in Android, creating a critical attack surface. We present Action Rebinding, a novel attack that allows a seemingly-benign app with zero dangerous permissions to rebind an agent's execution. By exploiting the inevitable observation-to-action gap inherent in the agent's reasoning pipeline, the attacker triggers foreground transitions to rebind the agent's planned action toward the target app. We weaponize the agent's task-recovery logic and Android's UI state preservation to orchestrate programmable, multi-step attack chains. Furthermore, we introduce an Intent Alignment Strategy (IAS) that manipulates the agent's reasoning process to rationalize UI states, enabling it to bypass verification gates (e.g., confirmation dialogs) that would otherwise be rejected. We evaluate Action Rebinding Attacks on six widely-used Android GUI agents across 15 tasks. Our results demonstrate a 100% success rate for atomic action rebinding and the ability to reliably orchestrate multi-step attack chains. With IAS, the success rate in bypassing verification gates increases (from 0% to up to 100%). Notably, the attacker application requires no sensitive permissions and contains no privileged API calls, achieving a 0% detection rate across malware scanners (e.g., VirusTotal). Our findings reveal a fundamental architectural flaw in current agent-OS integration and provide critical insights for the secure design of future agent systems. To access experimental logs and demonstration videos, please contact [e-mail].

Problem

• Zero-Permission Manipulation
• GUI Agents
• Visual Atomicity
• Action Rebinding
• Intent Alignment Strategy

Innovation

• Action Rebinding
• Visual Atomicity
• Intent Alignment Strategy
• GUI Agent Security
• Zero-Permission Attack