Few-Shot Action Recognition via Intra- and Inter-Video Information Maximization

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Few-shot action recognition leverages intra-video and inter-video information.
• Current methods inadequately exploit these information sources, leading to reduced efficiency and inaccurate relationships.
• Jointly considering both inter- and intra-video information is underexplored.

Purpose of the Study:

• To propose a novel framework, Video Information Maximization (VIM), for few-shot video action recognition.
• To enhance the utilization of intra-video information through adaptive sampling.
• To improve the precision of inter-video information measurement via action alignment.

Main Methods:

• VIM employs an adaptive spatial-temporal video sampler to select important frames and regions.
• A spatial-temporal action alignment model is used for feature-level alignment.
• A new training objective based on mutual information maximizes both intra- and inter-video information.

Main Results:

• The adaptive sampler preserves critical action information and reduces interference.
• Action alignment enables more precise inter-video similarity measurements.
• Experimental results on public datasets demonstrate VIM's effectiveness.

Conclusions:

• VIM effectively maximizes intra- and inter-video information for few-shot action recognition.
• The proposed methods address limitations in current action recognition techniques.
• VIM offers a promising direction for advancing few-shot video understanding.