UniCon-Former: Unified Convolution Transformer is All You Need for Hand Gesture Recognition

2026-08-13

AI Summary

This work addresses the limitations of conventional convolutional neural networks (CNNs) in capturing global contextual information for gesture recognition and the high computational cost of standard Transformers. To this end, the authors propose a pyramid architecture that effectively integrates convolutional and Transformer modules. Specifically, lightweight convolutional projections are introduced prior to each Transformer stage to compress input dimensions, enabling efficient multi-scale modeling. This design jointly preserves fine-grained local details and long-range global dependencies while significantly reducing both model parameters and computational complexity measured in multiply-accumulate operations (MACs). The proposed method achieves state-of-the-art accuracy on the NVGesture and Briareo benchmark datasets, demonstrating its effectiveness and efficiency for gesture recognition tasks.

Abstract

Convolutional Neural Networks (CNNs) capture local features efficiently but struggle with global context due to their limited receptive field. On the other hand, transformers effectively capture global dependencies through self-attention but suffer from high redundancy and computational costs. Thus, to leverage the advantages of both CNNs and transformers, we propose a unified model (UniCon-Former) that aims to provide robust and efficient performance on dynamic hand gesture recognition. The unified approach helps the model to learn both local and global features. At the beginning of each transformer stage, the convolution projections help in decreasing the dimension of the input vectors of the transformer block. This creates a pyramidal structure at each transformer stage. These features enable the UniCon-Former to reduce resource usage than vanilla transformers, making it flexible for learning multi-scale and high-resolution features, which is required in hand gesture recognition. We have performed experiments with NVGesture and Briareo datasets and achieved state-of-the-art results with fewer parameters and MACs.

Problem

• hand gesture recognition
• convolutional neural networks
• transformers
• global context
• local features

Innovation

• Convolution Transformer
• Hand Gesture Recognition
• Pyramidal Structure
• Multi-scale Feature Learning
• Efficient Attention