(A)I Sees What You Don't: Exploiting New Attack Surfaces in Third-Party Mobile Agents

2026-06-30

AI Summary

This work reveals a novel attack surface introduced by third-party mobile agents that rely on vision-language models (VLMs) to perceive screens and perform actions—an exposure absent in traditional applications. It systematically investigates the security risks stemming from discrepancies between human and machine vision, analyzing how agents interact with their environment to identify two primary attack vectors: screen perception and misuse channels. The study proposes seven VLM-specific attack techniques, including sub-visual text injection, exploitation of imperceptible pixels, screenshot manipulation, and host command injection. Experiments demonstrate successful arbitrary command execution without privileged permissions across five mainstream mobile agent frameworks, with attacks remaining entirely invisible to users. These findings expose a fundamental trust mismatch in current security models when confronted with perception-driven intelligent agents.

Abstract

Third-party mobile agents powered by Vision-Language Models (VLMs) have emerged as a promising paradigm for automating smartphone interactions. These agents act as high-privilege decision-makers, perceiving device states through screenshots and executing actions via VLM reasoning, transforming how an agent app interacts with the environment (i.e., other apps or the OS). Correspondingly, this transformation introduces new attack surfaces or transforms benign/harmless interfaces into exploitable ones for mobile devices. In this paper, we summarize key differences between third-party mobile agent apps and general apps when interacting with the environment, analyze the security posture of agents, and identify two unique attack surfaces compared to general mobile apps: the Screen Perception Attack Surface, which exploits the gap between human and machine vision, and the Misused Channel Attack Surface, which intercepts or manipulates the agent's execution pipeline. We design and implement seven concrete attacks, from subliminal text injection and invisible pixel zone exploitation to screenshot tampering and host PC command injection. Our evaluation of five popular mobile agent frameworks demonstrates that a malicious app can hijack agent actions and achieve arbitrary command execution even without any privilege permissions, while remaining visually indistinguishable to users. These findings reveal a fundamental trust mismatch in autonomous agent design and highlight the urgent need for perception-aware security models on multi-tenant platforms.

Problem

• Third-party mobile agents
• Vision-Language Models
• Attack surfaces
• Screen perception
• Security

Innovation

• Vision-Language Models
• Mobile Agents
• Attack Surface
• Screen Perception
• Autonomous Security