Human Interaction Classification in Sliding Video Windows Using Skeleton Data Tracking and Feature Extraction.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Human activity recognition from video is crucial for applications like surveillance and human-robot interaction.
• Skeleton data provides a robust representation for human pose and motion analysis.
• Existing methods often struggle with accuracy and computational efficiency.

Purpose of the Study:

• To develop and evaluate a novel Long Short-Term Memory (LSTM)-based human activity classifier for skeleton data.
• To improve the accuracy and efficiency of two-person interaction classification.
• To investigate the impact of feature engineering on deep neural network performance for activity recognition.

Main Methods:

• Video analysis using a sliding window approach to create short-time chunks.
• Skeleton data estimation using software like OpenPose or HRNet, followed by correction.
• Knowledge-aware feature extraction from corrected skeleton data.
• Training and evaluation of single-, double-, and triple-channel LSTM network architectures.

Main Results:

• The most efficient LSTM model achieved 96% accuracy in two-person interaction classification on the NTU RGB+D dataset.
• Performance was competitive with state-of-the-art methods like Adaptive Graph Convolutional Networks (AGCN) and 3D Convolutional Networks.
• Cross-validation on the UT-Interaction dataset demonstrated the robustness of the sliding-window strategy for changing interactions.

Conclusions:

• A two-step approach combining skeleton feature engineering and deep neural networks offers a practical balance between accuracy and computational complexity.
• Early correction of imperfect skeleton data and knowledge-aware relational feature extraction are key to high performance.
• LSTM-based models show significant promise for skeleton-based human activity classification.