Hadaptive-Net: Efficient Vision Models via Adaptive Cross-Hadamard Synergy

2025-05-28

AI Summary

To address the weak inter-channel interaction, redundant channel expansion, and limited representational capacity of convolutional operations in lightweight vision models, this paper systematically investigates, for the first time, the potential of the Hadamard product for inter-channel modeling. We propose the Adaptive Channel-wise Hadamard (ACH) module, which enables dynamic inter-channel interaction and high-dimensional feature collaboration via adaptive weight learning. Integrated with a dynamic channel expansion mechanism and efficient tensor operations, ACH forms the lightweight architecture Hadaptive-Net. Extensive experiments demonstrate that Hadaptive-Net achieves state-of-the-art efficiency–accuracy trade-offs: it reduces model parameters by 32% and FLOPs by 41% compared to baseline lightweight models, while matching or exceeding the accuracy of leading architectures—including MobileNetV3 and EfficientNet-Lite—on image classification and object detection benchmarks. The method thus delivers superior inference speed without compromising accuracy.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Recent studies have revealed the immense potential of Hadamard product in enhancing network representational capacity and dimensional compression. However, despite its theoretical promise, this technique has not been systematically explored or effectively applied in practice, leaving its full capabilities underdeveloped. In this work, we first analyze and identify the advantages of Hadamard product over standard convolutional operations in cross-channel interaction and channel expansion. Building upon these insights, we propose a computationally efficient module: Adaptive Cross-Hadamard (ACH), which leverages adaptive cross-channel Hadamard products for high-dimensional channel expansion. Furthermore, we introduce Hadaptive-Net (Hadamard Adaptive Network), a lightweight network backbone for visual tasks, which is demonstrated through experiments that it achieves an unprecedented balance between inference speed and accuracy through our proposed module.

Problem

• Exploring Hadamard product's untapped potential in neural networks
• Developing efficient cross-channel interaction via Adaptive Cross-Hadamard
• Balancing speed and accuracy in lightweight vision models

Innovation

• Adaptive Cross-Hadamard for channel expansion
• Lightweight Hadaptive-Net for visual tasks
• Balances inference speed and accuracy