Multi-Scale Attention 3D Convolutional Network for Multimodal Gesture Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Hand information is critical for gesture recognition.
• Current keypoint-based methods are complex and prone to errors.
• Dynamic gesture recognition requires both spatial and temporal attention.

Purpose of the Study:

• To propose a multi-scale attention 3D convolutional network for enhanced gesture recognition.
• To address limitations of keypoint-based methods and incorporate multimodal data.
• To improve accuracy and efficiency in dynamic gesture recognition.

Main Methods:

• A multi-scale attention 3D convolutional network is proposed.
• Local attention focuses on hand regions using a hand detector.
• Dual spatiotemporal attention module provides global context.
• Multimodal fusion integrates RGB and depth data features.

Main Results:

• The proposed network achieves both local and global attention.
• Multimodal fusion effectively utilizes differences between data modalities.
• Experiments on Chalearn LAP and Briareo datasets demonstrate effectiveness.
• The method outperforms existing state-of-the-art approaches.

Conclusions:

• The multi-scale attention 3D convolutional network is effective for gesture recognition.
• The fusion of multimodal data enhances performance.
• The proposed method offers an efficient and accurate solution for dynamic gesture recognition.