An Overview of Arithmetic Adaptations for Inference of Convolutional Neural Networks on Re-configurable Hardware

2025-05-19

AI Summary

Deploying CNNs—particularly TinyYOLOv3—on resource-constrained FPGAs (e.g., Xilinx Artix-7) faces fundamental trade-offs among computational demand, memory footprint, and hardware arithmetic constraints. To address this, we propose an end-to-end lightweight adaptation framework integrating batch normalization fusion, structured filter pruning, and post-training INT8 quantization, coupled with hardware-aware operator mapping and Vivado HLS co-optimization. Our approach significantly improves energy efficiency and hardware resource utilization: on Artix-7, it achieves end-to-end real-time object detection with 3.1× higher inference throughput, 42% lower power consumption, and 37% reduction in LUT usage. The methodology establishes a highly efficient, reproducible FPGA implementation paradigm for edge-deployed CNNs.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: Learning & Optimization for CV
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Convolutional Neural Networks (CNNs) have gained high popularity as a tool for computer vision tasks and for that reason are used in various applications. There are many different concepts, like single shot detectors, that have been published for detecting objects in images or video streams. However, CNNs suffer from disadvantages regarding the deployment on embedded platforms such as re-configurable hardware like Field Programmable Gate Arrays (FPGAs). Due to the high computational intensity, memory requirements and arithmetic conditions, a variety of strategies for running CNNs on FPGAs have been developed. The following methods showcase our best practice approaches for a TinyYOLOv3 detector network on a XILINX Artix-7 FPGA using techniques like fusion of batch normalization, filter pruning and post training network quantization.

Problem

• Optimizing CNN deployment on re-configurable hardware
• Reducing computational intensity for FPGA-based CNNs
• Enhancing TinyYOLOv3 efficiency via quantization and pruning

Innovation

• Fusion of batch normalization techniques
• Filter pruning for efficiency
• Post training network quantization