Lightweight Semantic-Guided Neural Networks Based on Single Head Attention for Action Recognition

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Skeleton-based action recognition leverages graph convolutional networks (GCNs) by modeling human poses as graphs.
• Semantic-guided neural networks (SGNs) offer fast action recognition by hierarchically learning spatial-temporal features using GCNs.
• Existing SGNs have limitations in recognizing actions where local node dependencies are crucial due to their focus on global feature learning.

Purpose of the Study:

• To address the limitations of SGNs in capturing local feature dependencies for skeleton-based action recognition.
• To develop a novel model that enables real-time action recognition on resource-constrained devices.
• To enhance the performance of action recognition by integrating a single head attention mechanism.

Main Methods:

• Proposed a novel model, SGN-SHA, by combining a semantic-guided neural network (SGN) with a single head attention (SHA) mechanism.
• SHA is designed to overcome the limitations of SGNs in learning local features.
• The SGN-SHA model was evaluated on various benchmark datasets for action recognition.

Main Results:

• The SGN-SHA model demonstrated significantly reduced computational complexity compared to existing methods.
• Achieved performance comparable to existing SGN models and other state-of-the-art approaches.
• Successfully enabled real-time action recognition capabilities, particularly on low-end devices.

Conclusions:

• The proposed SGN-SHA model effectively integrates global and local feature learning for skeleton-based action recognition.
• Offers a computationally efficient solution for real-time action recognition, suitable for deployment on devices with limited resources.
• Represents a significant advancement in the field, balancing performance and efficiency.