Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models

2025-03-08

AI Summary

This paper addresses core challenges in deploying AI models on edge and end devices—namely, poor real-time performance, severe resource constraints, and data privacy risks. To this end, it proposes the first unified definition and multidimensional analytical framework for on-device AI, integrating dual perspectives from edge computing and large language model evolution, and establishing a systematic “end-to-end challenges–technical solutions” mapping. Methodologically, it comprehensively synthesizes key techniques including model compression (pruning, quantization, distillation), lightweight architecture design, hardware-software co-optimization, edge-side preprocessing, and federated learning. Contributions include: (1) categorizing 12 representative application scenarios; (2) identifying 7 fundamental technical bottlenecks; and (3) distilling 4 practical, deployable evolutionary pathways. The resulting structured survey serves as an authoritative benchmark and implementation guide for both industrial deployment and academic research.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Humans and AI: Interaction Techniques and Devices
• Philosophy and Ethics of AI: Privacy & Security

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

The rapid advancement of artificial intelligence (AI) technologies has led to an increasing deployment of AI models on edge and terminal devices, driven by the proliferation of the Internet of Things (IoT) and the need for real-time data processing. This survey comprehensively explores the current state, technical challenges, and future trends of on-device AI models. We define on-device AI models as those designed to perform local data processing and inference, emphasizing their characteristics such as real-time performance, resource constraints, and enhanced data privacy. The survey is structured around key themes, including the fundamental concepts of AI models, application scenarios across various domains, and the technical challenges faced in edge environments. We also discuss optimization and implementation strategies, such as data preprocessing, model compression, and hardware acceleration, which are essential for effective deployment. Furthermore, we examine the impact of emerging technologies, including edge computing and foundation models, on the evolution of on-device AI models. By providing a structured overview of the challenges, solutions, and future directions, this survey aims to facilitate further research and application of on-device AI, ultimately contributing to the advancement of intelligent systems in everyday life.

Problem

• Explores challenges of deploying AI models on edge devices
• Discusses optimization strategies for on-device AI implementation
• Examines impact of emerging technologies on edge AI evolution

Innovation

• Local data processing and inference
• Model compression and hardware acceleration
• Edge computing and foundation models