Multi-scale and attention enhanced graph convolution network for skeleton-based violence action recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Skeleton-based human action recognition is crucial for surveillance and safety.
• Existing Graph Convolutional Networks (GCNs) face challenges in balancing performance and parameter complexity.
• Violence action recognition requires sophisticated feature learning from complex human motion data.

Purpose of the Study:

• To develop a novel Multi-scale Attention Spatiotemporal Graph Convolutional Network (MSA-STGCN) for enhanced human violence action recognition.
• To improve recognition accuracy while reducing computational complexity.
• To leverage multi-modal skeleton data for robust feature extraction.

Main Methods:

• Preprocessing original joint data into four variants: joint position, bone vector, joint motion, and bone motion.
• Constructing a spatial multi-scale GCN with an attention mechanism for spatial feature extraction.
• Designing a temporal GCN with hybrid dilation convolution to capture multi-scale temporal context.
• Fusing information from multiple skeleton data streams to explore inter-stream relationships.

Main Results:

• Achieved 95.3% accuracy on the custom Filtered NTU RGB+D dataset with only 1.21M parameters.
• Attained 36.2% (Top-1) and 58.5% (Top-5) accuracy on the Kinetics Skeleton 400 dataset.
• Demonstrated effective violence action recognition without significant parameter increase.

Conclusions:

• The proposed MSA-STGCN framework effectively recognizes human violence actions.
• The method achieves state-of-the-art performance with reduced parameter complexity.
• Multi-modal skeleton data fusion enhances the robustness of action recognition.