A Static-Dynamic Composition Framework for Efficient Action Recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Dynamic inference adaptively allocates computational resources for efficient action recognition.
• Existing methods focus on data efficiency by reducing spatial/temporal redundancy, often using fixed, computationally expensive networks.

Purpose of the Study:

• Introduce a novel model-efficient regime for action recognition by dynamically selecting network components.
• Address network redundancy by dynamically selecting a partial network in real-time.

Main Methods:

• Propose the static-dynamic composition (SDCOM) framework with static and dynamic networks.
• Utilize a slimmable network mechanism and a novel meta-learning scheme for dynamic feature extraction.
• Dynamically activate minimal network width to extract supplementary features based on evaluated gaps.

Main Results:

• SDCOM accurately recognizes a large majority of frames (76%-92%) by combining primary and lightweight supplementary features.
• Achieve significant recognition efficiency, saving 90% of baseline floating point operations (FLOPs) on benchmark datasets.
• Demonstrate comparable or superior accuracy to state-of-the-art methods on ActivityNet, FCVID, and Mini-Kinetics.

Conclusions:

• The proposed SDCOM framework significantly enhances action recognition efficiency through model efficiency.
• Dynamically selecting network components offers a promising direction for efficient deep learning models.
• Balancing network efficiency and feature capacity is crucial for effective real-time video analysis.