Whole and Part Adaptive Fusion Graph Convolutional Networks for Skeleton-Based Action Recognition

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Spatiotemporal graph convolution has advanced skeleton-based action recognition.
• Existing methods often model the entire skeleton uniformly, neglecting regional movement variations and inter-part correlations.

Purpose of the Study:

• To develop novel graph convolution methods that capture unique features of different human skeleton parts and their correlations.
• To improve the accuracy and effectiveness of skeleton-based action recognition.

Main Methods:

• Proposed Whole Graph Convolution Network (WGCN) for whole-scale skeleton spatiotemporal features.
• Proposed Part Graph Convolution Network (PGCN) for part-scale skeleton spatiotemporal features by dividing the skeleton into subgraphs.
• Developed an adaptive fusion module to combine WGCN and PGCN features for enhanced representation.

Main Results:

• The integrated Whole and Part Graph Convolutional Neural Network (WPGCN) demonstrated superior performance.
• Outperformed state-of-the-art methods on the NTU RGB+D 60, NTU RGB+D 120, and Kinetics Skeleton 400 datasets.

Conclusions:

• The WPGCN effectively captures both global and local spatiotemporal features for skeleton-based action recognition.
• The proposed adaptive fusion approach enhances the discriminative power of skeleton features, leading to improved recognition accuracy.