Video-Based Human Activity Recognition Using Deep Learning Approaches

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Human activity recognition (HAR) is crucial for analyzing human behavior using sensor data.
• Recognizing actions in videos with multiple interacting entities requires advanced spatial modeling.
• Deep learning models offer powerful tools for visual reasoning in action recognition tasks.

Purpose of the Study:

• To evaluate and map the current state of human action recognition in RGB videos using deep learning.
• To assess the performance of Residual Network (ResNet) and Vision Transformer (ViT) architectures.
• To investigate the impact of semi-supervised learning and DINO (self-DIstillation with NO labels) on HAR.

Main Methods:

• Implemented and evaluated ResNet and ViT architectures with a semi-supervised learning approach.
• Utilized DINO (self-DIstillation with NO labels) to enhance model capabilities.
• Tested models on the Human Motion Database (HMDB51) benchmark for action recognition.

Main Results:

• The Vision Transformer (ViT) architecture demonstrated promising performance in video classification.
• A bi-dimensional ViT combined with Long Short-Term Memory (LSTM) achieved high accuracy on the HMDB51 dataset.
• The ViT-LSTM model achieved 96.7 ± 0.35% accuracy in training and 41.0 ± 0.27% in testing phases.

Conclusions:

• Deep learning models, particularly Vision Transformers, show significant potential for complex human action recognition.
• Semi-supervised learning and DINO enhance the effectiveness of HAR models.
• The proposed ViT-LSTM architecture provides a robust solution for video-based human action recognition.