Geometric Deep Neural Network Using Rigid and Non-Rigid Transformations for Landmark-Based Human Behavior Analysis.

Area of Science:

• Computer Vision
• Machine Learning
• Geometric Deep Learning

Background:

• Deep learning models often assume Euclidean data structures, which may not apply to pre-processed motion data residing in non-linear spaces.
• Analyzing human motion using landmarks requires methods that can handle complex, non-linear data configurations.

Purpose of the Study:

• To propose KShapenet, a novel geometric deep learning framework for 2D and 3D landmark-based human motion analysis.
• To address the limitations of Euclidean deep learning architectures for non-linear motion data.

Main Methods:

• Modeling landmark configuration sequences as trajectories on Kendall's shape space.
• Mapping shape space trajectories to a linear tangent space for structured data representation.
• Employing a deep learning architecture with a layer optimizing rigid/non-rigid transformations, followed by a CNN-LSTM network.

Main Results:

• KShapenet demonstrated competitive performance on 3D human landmark sequences for action and gait recognition.
• The approach achieved state-of-the-art results for 2D facial landmark sequences in expression recognition.

Conclusions:

• Geometric deep learning, specifically KShapenet, offers a powerful alternative for analyzing complex human motion data.
• The method effectively handles non-linearities in landmark data, advancing the field of human motion analysis.