Video-based hand gesture recognition via SPD manifold spatial representation and optical flow motion features

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Machine Learning

Background:

• Accurate hand gesture recognition is crucial for intuitive human-computer interaction.
• Existing methods struggle to model both spatial and temporal dynamics effectively.
• A unified approach is needed to capture the complexity of evolving hand gestures.

Purpose of the Study:

• To develop a unified feature representation framework for dynamic hand gesture recognition.
• To effectively combine spatial and temporal information for improved performance.
• To address the limitations of existing methods focusing on either spatial or temporal features.

Main Methods:

• Proposed a novel framework integrating spatial descriptors on the Symmetric Positive Definite (SPD) manifold and temporal motion features.
• Utilized grid-based optical flow histograms for temporal feature extraction.
• Mapped SPD spatial descriptors to Euclidean space via the Log-Euclidean metric for feature fusion.

Main Results:

• The unified representation captured complementary spatial and temporal information effectively.
• Achieved superior classification performance compared to using spatial or temporal features alone.
• Attained 99.31% accuracy on the Cambridge Hand Gesture dataset and 97.23% on the Northwestern University Hand Gesture dataset.

Conclusions:

• Integrating manifold-aware spatial features with motion-based temporal cues offers a robust solution.
• The proposed framework enhances dynamic hand gesture recognition accuracy.
• This approach provides a practical and effective method for advanced human-computer interaction.