Spatiotemporal Decouple-and-Squeeze Contrastive Learning for Semisupervised Skeleton-Based Action Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Contrastive learning is effective for semi-supervised skeleton-based action recognition.
• Existing methods often mix spatiotemporal information, hindering the learning of frame- and joint-level specific semantics.

Purpose of the Study:

• To propose a novel framework, spatiotemporal decouple-and-squeeze contrastive learning (SDS-CL), for learning richer skeleton-based action representations.
• To address the limitations of existing methods by decoupling and contrasting spatiotemporal information at different granularities.

Main Methods:

• Developed a spatiotemporal-decoupling intra-inter attention (SIIA) mechanism to capture spatiotemporal-specific information.
• Introduced novel contrastive losses: spatial-squeezing temporal-contrasting loss (STL), temporal-squeezing spatial-contrasting loss (TSL), and global-contrasting loss (GL).
• Jointly contrasted spatial-squeezing, temporal-squeezing, and global features at frame, joint, and skeleton levels, respectively.

Main Results:

• The proposed SDS-CL framework achieved significant performance gains on four public datasets.
• Demonstrated superior performance compared to other competitive skeleton-based action recognition methods.
• The decoupling of spatiotemporal information led to more comprehensive action representations.

Conclusions:

• The SDS-CL framework effectively learns enhanced action representations by leveraging decoupled spatiotemporal features.
• The proposed attention mechanism and contrastive losses contribute to improved performance in semi-supervised skeleton-based action recognition.
• SDS-CL offers a promising direction for future research in human action recognition.