A Deep Sequence Learning Framework for Action Recognition in Small-Scale Depth Video Dataset.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Deep learning models for human action recognition typically rely on RGB or skeleton data, with depth video models being less common.
• Training deep models with limited data is a significant challenge in depth-based action recognition research.
• Existing methods often struggle with the scarcity of depth video datasets.

Purpose of the Study:

• To propose a novel deep learning model for direct depth video sequence classification.
• To address the challenge of limited training data in depth-based human action recognition.
• To improve the efficacy and performance of action recognition using depth data.

Main Methods:

• A four-stream representation is created by transforming depth videos into multi-view temporal motion frames.
• DenseNet121 with ImageNet pre-trained weights extracts frame-level features from depth and motion streams.
• Features are processed through bi-directional Long Short-Term Memory (BLSTM) networks and multi-head self-attention (MHSA) for temporal analysis and correlation capture.

Main Results:

• The proposed framework demonstrates efficacy on small-scale depth datasets (MSRAction3D, DHA).
• The model achieves superior performance compared to existing depth data-based action recognition methods.
• Effective action classification is achieved even with insufficient training samples.

Conclusions:

• The developed deep model offers a viable solution for human action recognition using depth video sequences, especially in data-limited scenarios.
• The four-stream representation and attention mechanisms enhance the model's ability to capture complex temporal dynamics.
• This work advances the field of depth-based action recognition by providing a robust and efficient framework.