Channel-Wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Graph convolutional networks (GCNs) are effective for skeleton-based action recognition.
• Existing GCNs face challenges in dynamically learning topologies and aggregating joint features across different channels.

Purpose of the Study:

• To propose a channel-wise topology refinement-graph convolution (GC) for dynamic topology learning and effective joint feature aggregation.
• To enhance the representation capability of GCNs for skeleton-based action recognition.

Main Methods:

• Introduced a channel-wise topology refinement-GC that learns a shared topology and refines it using channel-specific joint correlations.
• Developed a channel-wise topology refinement squeeze-excitation transformer to model long-range joint dependencies and adjust channel-wise feature weights.
• Utilized channel-attention mechanisms and global information aggregation in temporal and spatial dimensions.

Main Results:

• The proposed method significantly reduces the difficulty in modeling channel-wise topologies with minimal extra parameters.
• The channel-wise topology refinement-GC demonstrates stronger representation capability by relaxing strict GCN constraints.
• Experimental validation on NTU RGB+D, NTU RGB+D120, and NW-UCLA datasets confirmed the effectiveness of the proposed approach.

Conclusions:

• The channel-wise topology refinement-GC and the associated transformer effectively improve skeleton-based action recognition.
• The proposed method offers a more robust and adaptable approach to learning human actions from skeletal data.