Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Skeleton-based human action recognition is crucial for understanding human activities.
• Recurrent neural networks (RNNs) have been used to model temporal dependencies in skeletal data.
• Existing methods often focus on either spatial or temporal domains, but not both simultaneously.

Purpose of the Study:

• To develop an advanced method for skeleton-based human action recognition.
• To simultaneously analyze both spatial and temporal information within skeletal data.
• To improve the robustness and accuracy of action recognition models.

Main Methods:

• Proposed a novel framework that extends recurrent neural networks to analyze spatial and temporal domains concurrently.
• Introduced a tree-structure based traversal framework utilizing the pictorial structure of Kinect skeletal data.
• Developed a new gating mechanism within the Long Short-Term Memory (LSTM) module to handle noisy skeletal data and learn data reliability.
• Implemented a multi-modal feature fusion strategy within the LSTM unit.

Main Results:

• The proposed method demonstrated significant effectiveness across seven challenging benchmark datasets for human action recognition.
• The novel gating mechanism successfully addressed noise in skeletal data, improving the learning of sequential information.
• Simultaneous spatial and temporal analysis yielded superior action recognition performance compared to existing methods.

Conclusions:

• The integrated spatial-temporal analysis framework significantly advances skeleton-based human action recognition.
• The proposed LSTM gating mechanism and feature fusion strategy enhance model robustness and accuracy.
• The method shows strong potential for real-world applications requiring reliable human activity analysis from skeletal data.