Multilevel Spatial-Temporal Excited Graph Network for Skeleton-Based Action Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Skeleton-based action recognition is crucial for understanding human motion.
• Existing methods struggle with complex joint connections due to limitations in spatial-temporal modeling and fixed topological structures.

Purpose of the Study:

• To propose a novel multilevel spatial-temporal excited graph network (ML-STGNet) to overcome the limitations of previous approaches.
• To enhance the exploration of joint connections in both spatial and temporal dimensions for improved action recognition.

Main Methods:

• Developed a multilevel graph convolution (ML-GCN) network using joint-level, part-level, and body-level graphs to model hierarchical human body relations.
• Introduced a spatial data-driven excitation (SDE) module for flexible and data-dependent learning of diverse joint relations.
• Designed an efficient temporal motion excitation (TME) module using temporal difference to emphasize motion-sensitive features.
• Incorporated a simplified multiscale temporal convolution (MS-TCN) network to enrich temporal feature representation.

Main Results:

• ML-STGNet demonstrated significant improvements over existing state-of-the-art methods.
• The proposed decoupling approach enhanced model flexibility and adaptability to various data samples.
• The network effectively captured complex joint connections in intricate human motions.

Conclusions:

• ML-STGNet offers a more robust and flexible framework for skeleton-based action recognition.
• The novel spatial and temporal modeling strategies lead to considerable performance gains.
• This approach advances the field by better capturing the nuances of human movement from skeleton data.