🤖 AI Summary
Deploying quantized neural networks (QNNs) on resource-constrained microcontrollers entails fundamental trade-offs among accuracy degradation, computational overhead, and memory constraints. This paper adopts a hardware–software co-design perspective to systematically analyze quantization mechanisms for embedded deployment, integrating cross-layer optimizations: model-level (e.g., asymmetric and per-channel quantization), software-level (TinyML framework adaptation and operator optimization), and hardware-level (dedicated accelerator support). We propose a QNN deployment methodology that jointly optimizes accuracy, efficiency, and framework/hardware compatibility. We empirically evaluate integration bottlenecks across mainstream TinyML frameworks—including TensorFlow Lite Micro and Apache TVM—and target hardware platforms such as ARM Cortex-M and RISC-V cores. Finally, we identify promising research directions, including ultra-low-bit quantization, structured sparsity, and compiler-aware training. Our work delivers a reproducible technical pathway and practical guidelines for robust TinyML deployment in real-world embedded systems.
📝 Abstract
The deployment of Quantized Neural Networks (QNNs) on resource-constrained devices, such as microcontrollers, has introduced significant challenges in balancing model performance, computational complexity and memory constraints. Tiny Machine Learning (TinyML) addresses these issues by integrating advancements across machine learning algorithms, hardware acceleration, and software optimization to efficiently run deep neural networks on embedded systems. This survey presents a hardware-centric introduction to quantization, systematically reviewing essential quantization techniques employed to accelerate deep learning models for embedded applications. In particular, further emphasis is put on critical trade-offs among model performance and hardware capabilities. The survey further evaluates existing software frameworks and hardware platforms designed specifically for supporting QNN execution on microcontrollers. Moreover, we provide an analysis of the current challenges and an outline of promising future directions in the rapidly evolving domain of QNN deployment.