SelfGCN: Graph Convolution Network With Self-Attention for Skeleton-Based Action Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Graph Convolutional Networks (GCNs) excel in skeleton-based action recognition but struggle with long-range dependencies and fixed skeleton topologies.
• Existing GCN methods often use uniform skeleton structures, limiting feature learning capabilities.

Purpose of the Study:

• To address limitations in GCNs for skeleton-based action recognition.
• To propose a novel architecture, SelfGCN, that captures both local and global dependencies and adapts to frame-specific spatial features.

Main Methods:

• Introduced the Graph Convolution Network with Self-Attention (SelfGCN).
• Developed a mixing features across self-attention and graph convolution (MFSG) module for parallel local and global relationship modeling.
• Incorporated a temporal-specific spatial self-attention (TSSA) module to learn frame-level spatial relationships.

Main Results:

• SelfGCN achieved state-of-the-art performance on the NTU RGB+D, NTU RGB+D120, and Northwestern-UCLA benchmark datasets.
• Demonstrated superior accuracy in skeleton-based action recognition compared to existing methods.
• Experimental results confirm the effectiveness of the proposed MFSG and TSSA modules.

Conclusions:

• SelfGCN effectively models both local and global dependencies in skeleton data.
• The proposed architecture significantly advances the accuracy of skeleton-based action recognition.
• SelfGCN offers a promising approach for complex human action understanding.