Action Keypoint Network for Efficient Video Recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Video recognition models require efficient methods to reduce redundancy.
• Existing dynamic methods often focus on spatial or temporal selection independently.
• Redundancies in videos are typically both spatial and temporal, and informative content can have diverse distributions.

Purpose of the Study:

• To propose an integrated approach for spatial and temporal selection in video recognition.
• To develop a network that identifies informative points in arbitrary shapes.
• To transform video recognition into an efficient point cloud classification task.

Main Methods:

• The Action Keypoint Network (AK-Net) integrates temporal and spatial selection.
• Informative 'action keypoints' are selected from feature maps using self-attention.
• Video recognition is reformulated as a point cloud classification problem using 1D kernels.

Main Results:

• AK-Net effectively selects informative content within arbitrary shapes.
• The method enhances efficiency in modeling spatial-temporal dependencies.
• Experimental results demonstrate consistent improvements in efficiency and performance.

Conclusions:

• AK-Net offers a novel approach to video recognition by combining spatial-temporal keypoint selection and point cloud classification.
• The proposed method significantly boosts efficiency and performance compared to baseline models.
• AK-Net addresses limitations of existing methods by handling diverse content distributions and simultaneous spatial-temporal redundancies.