Lightweight and efficient skeleton-based sports activity recognition with ASTM-Net.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Human Activity Recognition (HAR) is crucial for video understanding, with skeletal data offering robustness against environmental variations.
• Existing Graph Convolutional Network (GCNN) methods struggle with dynamic spatial relationships and integrating spatial-temporal information for complex actions.

Purpose of the Study:

• To introduce ASTM-Net, a novel Activity-aware SpatioTemporal Multi-branch graph convolutional network designed to overcome limitations in skeletal HAR.
• To enhance the capture of dynamic node affinities and the interplay between spatial and temporal features for improved action recognition.

Main Methods:

• Developed the Activity-aware Spatial Graph convolution Module (ASGM) to dynamically model Activity-Aware Adjacency Graphs (3A-Graphs) by fusing multiple graph types.
• Introduced the Temporal Multi-branch Graph convolution Module (TMGM) using parallel branches with dilated convolutions and pooling for efficient temporal modeling.
• Integrated ASGM and TMGM to jointly capture spatio-temporal information with reduced computational complexity.

Main Results:

• ASTM-Net achieved superior performance on benchmark datasets (NTU-RGB+D, NTU-RGB+D 120, Toyota Smarthome), outperforming state-of-the-art methods.
• Demonstrated significant reductions in parameters (51.9%) and FLOPs (49.7%) compared to MST-GCNN-ALLs, while improving accuracy by 0.82%.
• Showcased high accuracy (86.94%) under 30% random node occlusion, indicating robustness.

Conclusions:

• ASTM-Net effectively addresses the limitations of existing GCNNs in skeletal HAR by dynamically modeling spatial affinities and temporal dependencies.
• The proposed multi-branch architecture offers a parameter-efficient and computationally effective solution for complex activity recognition tasks.
• ASTM-Net represents a significant advancement in skeletal-based Human Activity Recognition, offering improved accuracy, efficiency, and robustness.