Toward Stronger 3D Human Action Representation Learning Via Efficient Spatiotemporal Decoupling

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• 3D human action understanding is crucial but challenging.
• Effective representation learning is key for advanced models.
• Existing methods struggle with robust and efficient learning.

Purpose of the Study:

• To develop a superior self-supervised framework for 3D human action representation.
• To enhance the contrastive learning paradigm for better performance.
• To improve robustness, efficiency, and generalisation ability.

Main Methods:

• Introduced Stronger Spatiotemporal Decoupling Network (Stronger-SCD-Net).
• Utilized deformation-driven data augmentation with a mask-out strategy.
• Redesigned the encoder architecture into a compact dual-branch model.
• Incorporated a global anchor representation for alignment and separation.
• Applied offline knowledge distillation for performance boost.

Main Results:

• Achieved superior performance on NTU-RGB+D and PKU-MMD datasets.
• Demonstrated improved generalisation ability and training efficiency.
• Showcased effectiveness across action recognition, retrieval, transfer, and semi-supervised learning.

Conclusions:

• Stronger-SCD-Net offers an effective self-supervised approach for 3D human action representation.
• The proposed methods significantly advance the state-of-the-art in action understanding.
• The framework provides a robust and interpretable solution for diverse downstream tasks.