Harvesting AI Computation at the Edge via Generic Approximation

2026-06-28

AI Summary

This work addresses the inefficient utilization of edge AI accelerators, which often remain underutilized due to redundant computations, while general-purpose processors struggle with compute-intensive tasks, leading to imbalanced resource usage. To bridge this gap, the authors propose an automatic task transformation mechanism based on representative neural architecture search (NAS), which approximates conventional computational tasks as lightweight neural networks. Coupled with a runtime scheduler, this approach opportunistically reuses idle AI accelerator cycles without interfering with primary AI workloads. Evaluated on a representative AIoT processor, the method significantly enhances performance across diverse edge tasks, effectively offloads computation from general-purpose processors, and achieves synergistic optimization of heterogeneous computing resources.

Abstract

With the widespread adoption of AI in various IoT scenarios such as smart sensing and processing, AI chips have become a common component at the edge. These chips are typically specialized for structured neural network (NN) processing and are designed to meet peak workload demands. However, they are often underutilized and suffer from considerable computational waste due to temporal or spatial redundancy in processing. Conversely, general-purpose processing engines at the edge may struggle with compute-intensive tasks such as signal processing and complex numerical operations because of stringent resource constraints. To address this imbalance, we propose a framework that harvests unused AI computation resources using general-purpose approximation techniques. The core idea is to automatically convert traditional computing tasks into neural network models via a representative neural architecture search (NAS) method. These approximate versions of general-purpose tasks are then deployed on AI engines during their idle periods. Specifically, we introduce a runtime scheduler that offloads these tasks to AI chips without compromising the performance of primary AI workloads, thereby alleviating the burden on general-purpose processors. Experiments on a representative AIoT processor show that our proposed AI computation harvesting strategy delivers substantial performance improvements across a set of edge processing tasks.

Problem

• edge computing
• AI chip underutilization
• computational waste
• resource-constrained processing
• task offloading

Innovation

• computation harvesting
• neural architecture search
• approximate computing
• edge AI
• task offloading