Generalized Few-Shot Video Classification With Video Retrieval and Feature Generation.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Few-shot learning (FSL) excels in image recognition but remains underdeveloped for video classification.
• Existing methods often overlook the critical role of robust video feature learning.
• Spatiotemporal feature extraction is crucial for accurate video understanding.

Purpose of the Study:

• To advance few-shot video classification by developing effective spatiotemporal feature learning techniques.
• To introduce novel approaches for few-shot video classification that reduce reliance on labeled data.
• To create more realistic benchmarks for evaluating few-shot and generalized few-shot video classification.

Main Methods:

• A two-stage approach involving 3D Convolutional Neural Networks (CNNs) for spatiotemporal feature learning on base classes.
• Leveraging tag-labeled videos and visual similarity for data augmentation in few-shot learning.
• Employing generative adversarial networks (GANs) to synthesize video features from semantic embeddings for novel classes.

Main Results:

• The baseline 3D CNN approach significantly outperformed prior methods by over 20 points on existing benchmarks.
• Novel methods utilizing tag retrieval and GANs further boosted performance, especially on new, more realistic benchmarks.
• The proposed methods demonstrated substantial improvements in both few-shot and generalized few-shot learning scenarios.

Conclusions:

• Effective spatiotemporal feature learning using 3D CNNs is vital for advancing few-shot video classification.
• Leveraging external data and generative models offers promising avenues to overcome data limitations in few-shot video tasks.
• The developed benchmarks provide a more comprehensive evaluation of FSL algorithms in realistic settings.