Dedicated Inference Engine and Binary-Weight Neural Networks for Lightweight Instance Segmentation

2024-06-17

2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

AI Summary

Instance segmentation on edge devices faces challenges in balancing accuracy, model size, and hardware efficiency. Method: This paper proposes a co-optimization framework for binary-weight neural networks (BNNs), comprising (i) a dual-mode ASIC inference engine specialized for BNNs—supporting inference via bitwise operations and additions only—and (ii) a lightweight fused architecture integrating SegNeXt and SparseInst. Contribution/Results: We introduce the first dual-mode BNN inference architecture and achieve a hardware implementation requiring only 52% of the area of conventional MAC-based units. Evaluated on the Person class, our method surpasses YOLACT in accuracy while reducing model size to 1/77.7 and cutting hardware resource usage by 48%. The design significantly improves energy efficiency and deployment feasibility for edge instance segmentation.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: Segmentation
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Cloud, edge and content delivery systems for the Web
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

Binary-weight Neural Networks (BNNs), in which weights are binarized and activations are quantized, are employed to reduce computational costs of various kinds of applications. In this paper, a design methodology of hardware architecture for inference engines is proposed to handle modern BNNs with two operation modes. Multiply-Accumulate (MAC) operations can be simplified by replacing multiply operations with bitwise operations. The proposed method can effectively reduce the gate count of inference engines by removing a part of computational costs from the hardware system. The architecture of MAC operations can calculate the inference results of BNNs efficiently with only 52% of hardware costs compared with the related works. To show that the inference engine can handle practical applications, two lightweight networks which combine the backbones of SegNeXt and the decoder of SparseInst for instance segmentation are also proposed. The output results of the lightweight networks are computed using only bitwise operations and add operations. The proposed inference engine has lower hardware costs than related works. The experimental results show that the proposed inference engine can handle the proposed instance-segmentation networks and achieves higher accuracy than YOLACT on the "Person" category although the model size is 77.7× smaller compared with YOLACT.

Problem

• Binary-Weight Neural Networks
• Object Segmentation
• Embedded Systems

Innovation

• Binary-Weight Neural Networks
• Hardware Architecture
• Efficient Inference