Spatio-Temporal Attention-Based LSTM Networks for 3D Action Recognition and Detection

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Human action analytics is crucial in computer vision for understanding human behavior.
• Extracting discriminative spatio-temporal features is essential for accurate action modeling.

Purpose of the Study:

• To propose a spatial and temporal attention model for human action recognition and detection from skeleton data.
• To enhance the extraction of discriminative spatio-temporal features for improved action analysis.

Main Methods:

• Utilized recurrent neural networks with long short-term memory units.
• Developed a spatial and temporal attention mechanism to focus on salient joints and frames.
• Implemented a regularized cross-entropy loss and joint training strategy for effective network training.
• Introduced a temporal attention-based method for generating action temporal proposals for detection.

Main Results:

• The model effectively focuses on discriminative joints within frames and assigns varying attention levels to different frames.
• Demonstrated effectiveness in both human action recognition and action detection tasks.
• Achieved strong performance on benchmark datasets including SBU Kinect Interaction, NTU RGB + D, and PKU-MMD.

Conclusions:

• The proposed spatial and temporal attention model significantly improves human action recognition and detection.
• The model's ability to selectively focus on spatio-temporal features offers a robust approach to action analysis.
• The findings highlight the potential of attention mechanisms in skeleton-based action understanding.