Global Co-Occurrence Feature and Local Spatial Feature Learning for Skeleton-Based Action Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Skeleton-based action recognition is crucial for human-computer interaction.
• Graph Convolutional Networks (GCNs) have advanced this field but struggle with global joint co-occurrence and local bone structure features.
• Unrelated channels can negatively impact model performance.

Purpose of the Study:

• To propose a novel model, Global Co-occurrence and Local Spatial feature learning (GCLS), to address limitations in current GCN-based action recognition.
• To improve the capture of global co-occurrence features among joints.
• To extract local spatial structure features and mitigate the influence of irrelevant channels.

Main Methods:

• GCLS utilizes a two-branch architecture: Vertex Attention Mechanism (VAM) branch for global features and Cross-kernel Feature Fusion (CFF) branch for local features and channel restraint.
• VAM-branch focuses on capturing comprehensive co-occurrence patterns across all joints.
• CFF-branch extracts features from adjacent bones and filters irrelevant channels.

Main Results:

• The proposed GCLS model demonstrated superior performance on large-scale datasets (NTU-RGB+D and Kinetics).
• GCLS effectively captures both global action co-occurrence and local spatial structures.
• The model shows significant improvements compared to existing mainstream approaches.

Conclusions:

• The GCLS model offers an effective solution for skeleton-based action recognition by integrating global and local feature learning.
• The proposed architecture successfully addresses the limitations of traditional GCN methods.
• GCLS represents a significant advancement in achieving higher accuracy in action recognition tasks.