A Dynamic Frame Selection Framework for Fast Video Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Video recognition models often process redundant frames, leading to high computational costs.
• Efficient video analysis requires methods that can adaptively focus on informative frames.

Purpose of the Study:

• To introduce AdaFrame, a conditional computation framework for adaptive frame selection in video recognition.
• To minimize computational cost in video recognition without sacrificing accuracy.

Main Methods:

• AdaFrame utilizes a Long Short-Term Memory (LSTM) with global memory as an agent to interact with video sequences.
• The framework employs policy search methods for training, enabling adaptive lookahead inference based on predicted utilities.
• It computes predictions, determines next observation points, and estimates future rewards at each time step.

Main Results:

• AdaFrame achieved comparable performance to using all frames on FCVID and ActivityNet benchmarks with a ResNet-101 model.
• It required only an average of 8.21 frames on FCVID and 8.65 frames on ActivityNet.
• The framework demonstrated compatibility with modern 2D and 3D video recognition networks.

Conclusions:

• AdaFrame offers a computationally efficient solution for video recognition by adaptively selecting relevant frames.
• The learned frame selection strategy reflects prediction difficulty at both instance and class levels.
• This method enables significant computational savings while maintaining high accuracy in video analysis tasks.