An enhanced spatial-temporal graph convolution network with high order features for skeleton-based action...

Area of Science:

• Computer Vision
• Human Motion Analysis
• Machine Learning

Background:

• Skeleton-based action recognition is crucial for understanding human motion.
• Existing methods using 3D joint coordinates lack higher-order spatial-temporal information.
• Distinguishing fine-grained actions requires richer motion representations.

Purpose of the Study:

• To develop novel geometric features for joints, bones, and motion streams.
• To propose new Graph Convolutional Network (GCN) architectures for enhanced action recognition.
• To improve the accuracy and discriminative power of skeleton-based action recognition.

Main Methods:

• Introduced novel geometric features: multi-level spatial normalization, higher-order temporal derivatives, and bone-structure encoding (lengths, angles, anatomical distances).
• Proposed two architectures: Enhanced Multi-Stream AGCN (EMS-AGCN) with late fusion and Multi-Branch AGCN (MB-AGCN) with early adaptive fusion.
• Evaluated on the NTU-RGB+D 60 benchmark dataset.

Main Results:

• EMS-AGCN achieved 96.2% accuracy, surpassing state-of-the-art.
• MB-AGCN achieved 95.5% accuracy, also outperforming existing methods.
• The proposed geometric features and fusion mechanisms significantly improved action recognition performance.

Conclusions:

• Incorporating higher-order geometric features enhances the capture of subtle motion dynamics.
• Adaptive fusion mechanisms in GCN architectures are effective for skeleton-based action recognition.
• The developed approach represents a substantial advancement in the field of computer vision for human motion analysis.