Search papers, labs, and topics across Lattice.
The UniCon-Former model integrates convolutional neural networks (CNNs) and transformers to enhance hand gesture recognition by effectively capturing both local and global features. By employing convolution projections at the start of each transformer stage, the model reduces input dimensionality and creates a pyramidal structure that optimizes resource usage. Experimental results on the NVGesture and Briareo datasets demonstrate that UniCon-Former achieves state-of-the-art performance with fewer parameters and multiply-accumulate operations (MACs) compared to traditional models.
Achieving state-of-the-art hand gesture recognition with a unified model that combines the strengths of CNNs and transformers while using fewer resources is a game changer for real-time applications.
Convolutional Neural Networks (CNNs) capture local features efficiently but struggle with global context due to their limited receptive field. On the other hand, transformers effectively capture global dependencies through self-attention but suffer from high redundancy and computational costs. Thus, to leverage the advantages of both CNNs and transformers, we propose a unified model (UniCon-Former) that aims to provide robust and efficient performance on dynamic hand gesture recognition. The unified approach helps the model to learn both local and global features. At the beginning of each transformer stage, the convolution projections help in decreasing the dimension of the input vectors of the transformer block. This creates a pyramidal structure at each transformer stage. These features enable the UniCon-Former to reduce resource usage than vanilla transformers, making it flexible for learning multi-scale and high-resolution features, which is required in hand gesture recognition. We have performed experiments with NVGesture and Briareo datasets and achieved state-of-the-art results with fewer parameters and MACs.
