Action Recognition with 3D Residual Attention and Cross Entropy

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Human activity recognition (HAR) is crucial for applications like surveillance and human-computer interaction.
• Existing methods often struggle to effectively capture complex spatiotemporal dynamics in videos.

Purpose of the Study:

• To propose a novel deep learning model, the 3D residual attention network (3DRFNet), for enhanced human activity recognition.
• To improve the model's ability to learn discriminative spatiotemporal features from video data.

Main Methods:

• Developed a 3D residual attention network (3DRFNet) integrating channel and spatial attention mechanisms within a 3D ResNet framework.
• Incorporated Fast Fourier Convolution (FFC) to enhance temporal and spatial feature extraction.
• Utilized cross-entropy loss for model training and backpropagation.

Main Results:

• 3DRFNet achieved state-of-the-art (SOTA) performance on human action recognition tasks.
• Attained high accuracies of 91.7% on the HMDB-51 dataset and 98.7% on the UCF-101 dataset.
• Demonstrated superior recognition accuracy and robustness, particularly in capturing key behavioral features.

Conclusions:

• The proposed 3DRFNet effectively learns spatiotemporal representations for human activity recognition.
• Attention mechanisms and FFC significantly contribute to the model's enhanced performance.
• 3DRFNet offers a robust and accurate solution for video-based human action recognition.