Expressive Keypoints for Skeleton-Based Action Recognition via Progressive Skeleton Evolution

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Traditional skeleton-based human action recognition methods using coarse keypoints struggle with subtle actions.
• Increased computational cost is a challenge with fine-grained skeletal representations.

Purpose of the Study:

• To improve the discriminative ability of skeleton-based human action recognition models.
• To address the computational challenges of fine-grained skeletal data.
• To extend the approach to multi-person scenarios efficiently.

Main Methods:

• Proposing Expressive Keypoints for a fine-grained skeletal representation incorporating hand and foot details.
• Introducing the Progressive Skeleton Evolution (PSE) strategy with learnable mapping matrices for efficient keypoint downsampling and weighting.
• Utilizing a plug-and-play Instance Pooling module for multi-person action recognition.

Main Results:

• The proposed Expressive Keypoints and PSE strategy significantly improve efficiency while retaining benefits of fine-grained details.
• Experimental results on seven datasets demonstrate superior performance compared to state-of-the-art methods.
• The Instance Pooling module effectively extends the approach to multi-person scenarios without substantial computational increase.

Conclusions:

• The PSE-GCN method offers a superior and efficient solution for skeleton-based human action recognition.
• Fine-grained skeletal representations combined with efficient processing strategies enhance action recognition accuracy.
• The approach shows strong potential for real-world applications requiring precise human action understanding.