Adaptive Local Spatiotemporal Features from RGB-D Data for One-Shot Learning Gesture Recognition

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Extracting distinctive spatiotemporal features for gesture recognition is challenging due to noise and limited training samples.
• Existing methods struggle with the constraints of one-shot learning and empirical motion variations.

Purpose of the Study:

• To propose an adaptive local spatiotemporal feature (ALSTF) method for robust gesture recognition using fused RGB-D data.
• To overcome limitations of noise and few-shot learning in gesture analysis.

Main Methods:

• Adaptive extraction of motion regions of interest (MRoIs) using grayscale and depth velocity variance.
• Keypoint detection based on adaptive local constraints on depth and velocity.
• Fusion of multiple descriptors in extended gradient and motion spaces for feature representation.

Main Results:

• The proposed ALSTF method significantly reduces the impact of noise on feature extraction.
• Achieved higher performance in one-shot learning settings on benchmark datasets (ChaLearn, CAD-60, MSRDailyActivity3D).
• Demonstrated comparable accuracy to state-of-the-art methods in leave-one-out cross-validation.

Conclusions:

• The ALSTF method provides an effective solution for robust gesture recognition with limited data.
• Fused RGB-D data and adaptive feature extraction enhance the distinctiveness of spatiotemporal representations.
• The approach shows promise for real-world applications requiring accurate gesture analysis.