Semantic-aware Video Representation for Few-shot Action Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Few-shot action recognition methods often use 2D features and struggle with temporal modeling and text integration.
• Existing approaches require complex components and distance functions, limiting their effectiveness in fusing multimodal information.

Purpose of the Study:

• To propose a Semantic-Aware Few-Shot Action Recognition (SAFSAR) model that overcomes limitations of current methods.
• To achieve state-of-the-art performance in few-shot action recognition using a simplified approach.

Main Methods:

• Leveraging 3D feature extractors and an effective feature-fusion scheme with cosine similarity for classification.
• Introducing a novel scheme to encode textual semantics into video representations for adaptive fusion.
• Encouraging visual encoders to extract semantically consistent features through integrated text and video feature alignment.

Main Results:

• The SAFSAR model demonstrates superior performance compared to existing methods.
• Achieved significant improvements on five challenging few-shot action recognition benchmarks.
• Validated the effectiveness of using 3D features and a simplified fusion scheme.

Conclusions:

• The proposed SAFSAR model offers a simple yet effective solution for few-shot action recognition.
• Directly using 3D features with adaptive fusion outperforms complex methods.
• SAFSAR enables compact alignment and fusion of text and video features for enhanced performance.