Attention-Based Temporal Encoding Network with Background-Independent Motion Mask for Action Recognition

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Traditional Convolutional Neural Networks (CNNs) struggle with high-dimensional video data, complex human dynamics, and background interference.
• Capturing intricate motion patterns in videos remains a challenge for existing CNN architectures.

Purpose of the Study:

• To propose a novel framework, the attention-based temporal encoding network (ATEN) with a background-independent motion mask (BIMM), for improved video action recognition.
• To enhance the ability of deep learning models to accurately recognize actions in videos, especially under challenging conditions.

Main Methods:

• Developed a motion segmenting approach using boundary priors and minimal geodesic distance within a weighted graph.
• Introduced a dynamic contrast segmenting procedure for isolating moving objects in complex environments.
• Designed a background-independent motion mask (BIMM) to enhance moving objects by suppressing irrelevant background.
• Integrated a long-range attention mechanism within ATEN to focus on semantically vital frames, reducing temporal redundancy and highlighting discriminative motion information.

Main Results:

• The ATEN with BIMM framework achieved 94.5% accuracy on the HMDB51 dataset.
• The framework attained 70.6% accuracy on the UCF101 dataset.
• Experimental results demonstrate superior performance compared to several existing video action recognition methods.

Conclusions:

• The proposed ATEN with BIMM framework effectively addresses the limitations of traditional CNNs in video action recognition.
• The attention mechanism and BIMM significantly improve the model's ability to capture complex human actions by focusing on relevant motion and suppressing background noise.