LM-Scout: Analyzing the Security of Language Model Integration in Android Apps

2025-05-13

AI Summary

This work systematically uncovers security risks arising from large language model (LLM) integration in Android applications: attackers can bypass developer-imposed constraints—such as query frequency, input length, and topic restrictions—enabling unauthorized LLM invocations that jeopardize developer reputation and economic interests. To address this, the authors introduce the first taxonomy of LLM usage restrictions and propose LM-Scout, the first fully automated detection tool. LM-Scout combines manual reverse engineering, dynamic instrumentation, rule-driven API call-chain tracing, and proof-of-concept (PoC) generation. Evaluated on 2,950 mainstream Android apps, it precisely identifies 120 exploitable vulnerabilities; validation on 181 sampled apps confirms successful bypass of LLM access controls in 127 cases. The study further pinpoints root causes and proposes practical, deployable hardening strategies—establishing both theoretical foundations and engineering best practices for securing LLMs on mobile platforms.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Developers are increasingly integrating Language Models (LMs) into their mobile apps to provide features such as chat-based assistants. To prevent LM misuse, they impose various restrictions, including limits on the number of queries, input length, and allowed topics. However, if the LM integration is insecure, attackers can bypass these restrictions and gain unrestricted access to the LM, potentially harming developers' reputations and leading to significant financial losses. This paper presents the first systematic study of insecure usage of LMs by Android apps. We first manually analyze a preliminary dataset of apps to investigate LM integration methods, construct a taxonomy that categorizes the LM usage restrictions implemented by the apps, and determine how to bypass them. Alarmingly, we can bypass restrictions in 127 out of 181 apps. Then, we develop LM-Scout, a fully automated tool to detect on a large-scale vulnerable usage of LMs in 2,950 mobile apps. LM-Scout shows that, in many cases (i.e., 120 apps), it is possible to find and exploit such security issues automatically. Finally, we identify the root causes for the identified issues and offer recommendations for secure LM integration.

Problem

• Analyzing insecure Language Model integration in Android apps
• Bypassing LM usage restrictions to gain unauthorized access
• Developing automated tool to detect vulnerable LM usage

Innovation

• Automated tool LM-Scout detects LM vulnerabilities
• Taxonomy categorizes LM usage restrictions bypass
• Manual analysis reveals insecure LM integration