Momentum Contrastive Teacher for Semi-Supervised Skeleton Action Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Current semi-supervised skeleton action recognition relies on self-supervised training followed by supervised fine-tuning.
• Self-supervised learning prioritizes data representation over direct label classification.
• Existing methods face challenges with limited large-scale skeleton data.

Purpose of the Study:

• To introduce SkeleMoCLR, a novel pseudo-label-based model for semi-supervised skeleton action recognition.
• To leverage contrastive learning for transferring discriminative features from large vision-text models when skeleton data is scarce.
• To improve action recognition accuracy by integrating pseudo-labels into memory queues for enhanced representation differentiation.

Main Methods:

• Utilized MoCo v2 as a foundation, extending it into a teacher-student network with a momentum encoder.
• Employed contrastive learning to pre-train skeleton encoders using features from large vision-text models.
• Generated high-confidence pseudo-labels using a pretrained MoCo v2 teacher encoder to train the query encoder.
• Incorporated pseudo-labels into memory queues, sampling negative samples across different pseudo-label classes.
• Jointly optimized classification loss (labeled and pseudo-labeled data) and contrastive loss (unlabeled data).

Main Results:

• SkeleMoCLR demonstrated superior performance compared to existing methods on benchmark datasets (NTU-60, NTU-120, PKU-MMD, NW-UCLA).
• The proposed method effectively transfers discriminative action features from large vision-text models.
• Pseudo-label integration in memory queues improved representation differentiation and model accuracy.

Conclusions:

• SkeleMoCLR offers a powerful approach to semi-supervised skeleton action recognition by effectively combining pseudo-labeling and contrastive learning.
• The method addresses data scarcity by transferring knowledge from large-scale vision-text models.
• SkeleMoCLR achieves state-of-the-art results, highlighting the potential of pseudo-label semi-supervised and self-supervised learning.