Onboard Optimization and Learning: A Survey

2025-05-07

AI Summary

This work addresses critical challenges in edge AI—namely, high real-time inference latency, excessive training energy consumption, and heightened privacy and security risks—stemming from severe resource constraints on edge devices. Methodologically, it proposes a hardware-software co-designed on-device AI optimization framework, pioneering the integration of model compression (pruning, quantization, knowledge distillation), decentralized learning (federated learning augmented with secure multi-party computation), and approximate inference, jointly optimized with domain-specific AI accelerators. The key contributions include: (1) establishing a scalable, robust, and privacy-enhancing on-device AI deployment paradigm tailored to dynamic edge environments; and (2) defining a unified evaluation framework and technology roadmap that systematically identifies fundamental bottlenecks and emerging research directions. Collectively, this work advances both theoretical understanding and practical deployment of edge AI through innovations that bridge algorithmic efficiency, hardware acceleration, and privacy-preserving distributed learning.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Philosophy and Ethics of AI: Privacy & Security
• Computer Vision: Learning & Optimization for CV

Application Category

• Security and Privacy: Security and privacy of machine learning and AI applications
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Onboard learning is a transformative approach in edge AI, enabling real-time data processing, decision-making, and adaptive model training directly on resource-constrained devices without relying on centralized servers. This paradigm is crucial for applications demanding low latency, enhanced privacy, and energy efficiency. However, onboard learning faces challenges such as limited computational resources, high inference costs, and security vulnerabilities. This survey explores a comprehensive range of methodologies that address these challenges, focusing on techniques that optimize model efficiency, accelerate inference, and support collaborative learning across distributed devices. Approaches for reducing model complexity, improving inference speed, and ensuring privacy-preserving computation are examined alongside emerging strategies that enhance scalability and adaptability in dynamic environments. By bridging advancements in hardware-software co-design, model compression, and decentralized learning, this survey provides insights into the current state of onboard learning to enable robust, efficient, and secure AI deployment at the edge.

Problem

• Optimizing model efficiency for resource-constrained edge devices
• Reducing inference costs and accelerating real-time processing
• Ensuring privacy and security in decentralized learning systems

Innovation

• Optimizes model efficiency for resource-constrained devices
• Accelerates inference speed through hardware-software co-design
• Ensures privacy-preserving decentralized collaborative learning