Cross-Modal Contrastive Learning Network for Few-Shot Action Recognition.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Few-shot action recognition faces challenges due to limited labeled data, leading to overfitting and poor generalization.
• Existing methods struggle to extract discriminative features from scarce visual data for robust video understanding.

Purpose of the Study:

• To propose a novel Cross-Modal Contrastive Learning Network (CCLN) for effective few-shot action recognition.
• To address data scarcity and improve feature representation in limited-data scenarios.

Main Methods:

• A Prototypical Generative Adversarial Network (PGAN) synthesizes additional training samples to mitigate data scarcity and overfitting.
• A Cross-Modal Contrastive Learning Module (CCLM) leverages semantic information to learn discriminative feature representations.
• A Spatial-Temporal Enhancement Module (SEM) models both spatial context within frames and temporal dynamics across frames.

Main Results:

• The proposed CCLN significantly outperforms state-of-the-art methods on four benchmark datasets: Kinetics, UCF101, HMDB51, and SSv2.
• Synthesized samples effectively alleviate overfitting, and contrastive learning enhances class-level feature learning.
• The spatial-temporal module improves the modeling of crucial sequential information in videos.

Conclusions:

• The CCLN framework demonstrates superior performance in few-shot action recognition by effectively combining data augmentation, cross-modal learning, and spatio-temporal modeling.
• The proposed approach offers a robust solution for recognizing new actions with limited data, advancing the field of video understanding.