Neighbor-Guided Consistent and Contrastive Learning for Semi-Supervised Action Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Semi-supervised learning is established for image classification but underexplored for video action recognition.
• Existing methods like FixMatch struggle with video data due to reliance on single modalities and limited supervised signals.
• Insufficient motion information in RGB data hinders direct transfer of image-based methods to video.

Purpose of the Study:

• To propose a novel semi-supervised method, Neighbor-Guided Consistent and Contrastive Learning (NCCL), for video action recognition.
• To address limitations of existing methods, including insufficient supervised signals, long training times, and poor feature discriminability.
• To effectively utilize both RGB and temporal gradient modalities for improved action recognition.

Main Methods:

• Developed a teacher-student framework incorporating RGB and temporal gradient (TG) inputs.
• Integrated neighbor information as a self-supervised signal to enhance consistency and compensate for limited labeled data.
• Introduced a neighbor-guided category-level contrastive learning term to improve feature discriminability by minimizing intra-class and maximizing inter-class distances.

Main Results:

• Extensive experiments on four datasets demonstrate the effectiveness of NCCL.
• NCCL achieves superior performance compared to state-of-the-art methods in video action recognition.
• The proposed method offers significantly lower computational cost.

Conclusions:

• NCCL effectively addresses the challenges of semi-supervised video action recognition.
• The integration of neighbor information and contrastive learning leads to more discriminative features and efficient training.
• NCCL represents a significant advancement in semi-supervised learning for video analysis.