Multi-Head Attention-Based Framework with Residual Network for Human Action Recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Human action recognition (HAR) is crucial for applications like human-computer interaction and assistive robotics.
• Traditional HAR methods face challenges with temporal complexities, intra-class variability, and inter-class similarities, leading to inaccuracies.
• Robust and efficient HAR is needed for real-world deployment.

Purpose of the Study:

• To develop a deep learning framework for efficient and robust human action recognition.
• To address limitations of traditional methods in handling complex temporal patterns and variations.
• To enable real-time HAR for practical applications.

Main Methods:

• A deep learning framework combining ResNet-18 for spatial features and Bi-LSTM for temporal features.
• Integration of a multi-head attention mechanism to prioritize critical motion details.
• Implementation of a motion-based frame selection using optical flow for efficiency.

Main Results:

• Achieved 96.60% accuracy on the UCF-101 dataset, outperforming state-of-the-art methods.
• Operates at 222 frames per second (FPS), demonstrating high computational efficiency.
• Validated in real-world scenarios on the TIAGo mobile service robot.

Conclusions:

• The proposed framework offers a robust and efficient solution for human action recognition.
• The model effectively captures human actions with reduced frame dependency, suitable for real-time systems.
• Demonstrates practical applicability in assistive robotics and other real-world scenarios.